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Abstract  of  Dissertation  Presented  to  the  Graduate  School  of  the  University  of  Florida 
in  Partial  Fulfillment  of  the  Requirements  for  the  Degree  of  Doctor  of  Philosophy 

THE  EFFECTS  OF  NONANALYTIC  AND  ANALYTIC  THINKING 
IN  NEW  PRODUCT  FORECASTING 

By 

Lisa  E.  Bolton 
May  2000 

Chairman:  Joseph  W.  Alba 
Major  Department:  Marketing 

Judgmental  forecasting  is  an  inherent  component  of  new  product  development 
and  strategic  decision-making.  Managers  and  firms  often  struggle  with  these 
decisions,  as  evidenced  by  failure  rates  of  new  products  and  new  businesses.  This 
dissertation  investigates  judgmental  processes  that  may  bias  forecasts  in  a new 
product  development  context.  For  my  purposes,  judgments  are  classified  as 
nonanalytic  or  analytic.  Analytic  judgments  reflect  an  evenhanded  approach  based  on 
causal  reasoning.  Various  tools  exist  to  assist  analytic  judgment  (including 
accountability,  scenario  analysis,  counterfactual  reasoning,  and  task  decomposition). 
In  contrast,  nonanalytic  judgment  lacks  the  depth  and  evenhandedness  of  analytic 
judgment.  Examples  of  nonanalytic  thinking  include  imagining  scenarios  and 
reasoning  by  analogy. 

I propose  that  nonanalytic  processes  can  have  powerful  effects  on  subsequent 
predictions  and  that  the  effects  of  nonanalytic  thinking  in  forecasting  situations  are 

vii 


fairly  resistant  to  subsequent  analytic  thought.  In  addition,  the  effectiveness  of 
analytic  techniques  in  debiasing  judgment  may  depend  on  how  these  biases  were 
initially  created.  Thus,  understanding  the  judgmental  processes  involved  in 
nonanalytic  and  analytic  thought  and  their  mediating  cognitive  processes  is  critical. 
Assessing  the  effectiveness  of  analytic  tools  and  understanding  judgmental  processes 
will  provide  managers  with  insight  into  current  and  best  practice  in  judgmental 
forecasting.  In  addition,  these  results  will  also  generalize  to  consumers  domains, 
including  how  consumers  respond  to  new  product  concepts,  predict  their  own 
behavioral  intentions,  and  plan  for  their  own  futures. 

My  research  consists  of  a series  of  laboratory  studies  in  a (simplified)  new 
product  development  context.  These  studies  demonstrate  systematic  effects  of 
nonanalytic  thinking  on  judgment.  Scenarios  and  analogies  are  shown  to  have 
enduring  effects  on  judgment,  despite  subsequent  use  of  analytic  tools. 

Counterfactual  reasoning  failed  to  eliminate  scenario  and  analogy  effects  on 
judgment.  Stronger  debiasing  attempts  that  used  counter-scenarios,  counter-analogies 
and  decomposition  also  failed.  Instead,  some  evidence  suggests  that  subsequent 
efforts  to  be  analytic  after  nonanalytic  thinking  exacerbate  bias.  For  example, 
accountability  increased  scenario-induced  optimism,  and  the  presentation  of 
evenhanded  facts  may  have  polarized  judgment.  Overall,  these  findings  demonstrate 
the  robustness  of  nonanalytic  effects  on  judgment  and  raise  questions  about  the 
effectiveness  of  various  analytic  tools  discussed  in  the  literature. 


viii 


CHAPTER  1 
INTRODUCTION 


Predictions  are  ubiquitous.  They  occur  among  consumers  in  common 
everyday  situations  (‘Will  this  new  product  really  deliver  its  promised  benefits?’ 
‘How  often  and  for  how  long  will  I use  this  product  before  I need  another  one?’ 
‘How  much  will  I spend  this  week/month?’),  and  also  permeate  business  life  (‘Will 
this  new  product  be  successful?’  ‘How  will  this  new  market  or  channel  affect  my 
sales?’  ‘What  new  technologies  will  affect  my  business,  and  how?’  ‘Will  this 
strategic  alliance  benefit  my  firm?’).  Accurate  predictions  are  usually  functional, 
because  they  enable  people  to  adapt  to  future  events  and  sometimes  to  intervene  and 
change  the  future. 

Most  everyday  predictions  involve  human  judgment.  Interestingly,  surveys 
suggest  that  the  most  popular  business  forecasting  methods  are  based  on  intuitive 
judgment,  including  the  sales  force  composite,  jury  of  executive  opinion,  and  naive 
extrapolation  (Armstrong,  Brodie,  & McIntyre  1987;  Dalrymple  1987).  However, 
most  research  efforts  in  marketing  have  focused  on  developing  and  refining  new 
quantitative  forecasting  methods,  and  have  neglected  intuitive  methods  that  rely  on 
human  judgment.  This  research  attempts  to  redress  that  imbalance  by  investigating 
the  human  judgment  processes  that  underlie  forecasting  in  business.  In  particular, 
this  research  focuses  on  judgmental  forecasting  in  order  to  answer  the  following 
questions:  How  do  people  make  predictions?  Are  these  predictions  biased?  If  so, 


how  do  they  become  biased?  Can  people  correct  for  these  biases?  If  so,  how  do 
debiasing  techniques  work  and  do  they  vary  as  a function  of  the  cause  of  the  original 
bias? 

New  Product  Forecasting 

Judgmental  forecasting  may  be  used  in  a variety  of  managerial  and  consumer 
contexts.  From  a managerial  perspective,  entrepreneurs  and  new  product  managers 
may  rely  on  intuitive  judgment  to  forecast  new  business  or  new  product  success  and 
to  develop  strategic  initiatives  and  business  plans  for  an  uncertain  future.  From  a 
consumer  perspective,  judgmental  forecasts  may  be  used  in  personal  financial  and 
career  planning  and  in  consumer  budgeting  and  time  management.  This  research 
focuses  on  the  domain  of  new  product  innovation,  an  increasingly  important  driver  of 
sales  and  growth  for  Product  Development  and  Management  Association  (PDMA) 
firms  (Rosenau  1996).  New  product  forecasting  can  be  investigated  from  both  a 
consumer  perspective  (e.g.,  judging  new  products  in  concept  testing  and  focus 
groups,  and  in  buying  situations)  and  managerial  perspective  (e.g.,  screening  new 
product  ideas,  forecasting  the  success  of  new  products  and/or  the  impact  of  new 
technologies). 

New  product  forecasting  provides  an  interesting  opportunity  to  examine 
managerial  forecasting,  especially  in  the  early  stages  of  new  product  development 
that  lack  ’hard  data’  upon  which  to  base  predictions.  Indeed,  the  new  product  idea 
mortality  curve  for  PDMA  member  firms  indicates  that  about  70%  of  new  product 
ideas  are  screened  out  before  reaching  the  testing  or  launch  stages  (Rosenau  1996). 
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The  new  product  context  has  several  important  characteristics  (also  shared  by  the 
other  domains  identified  above)  that  may  increase  reliance  on  judgmental  forecasting. 
First,  new  products  tend  to  be  viewed  as  relatively  unique,  so  historical  data  on  other 
new  products  may  be  neglected.  Second,  new  product  forecasting  is  a complex  task 
and  managers  may  prefer  holistic  evaluation  of  a single  identified  new  product 
alternative  rather  than  an  evaluation  across  the  (potentially  infinite)  set  of  other 
alternatives.  Both  of  these  characteristics  may  favor  judgmental  rather  than 
quantitative  methods  and  nonanalytic  rather  than  analytic  processes.  Third,  new 
product  concepts  may  shift  from  relatively  abstract  ambiguous  ideas  in  the  very  early 
stages  to  relatively  specific  prototypes  in  the  later  stages  of  development.  In 
conjunction,  judgmental  processes  may  also  shift.  Fourth,  managers  may  become 
quite  personally  involved  in  new  product  development  projects,  which  may  heighten 
biases  due  to  desirability  and  controllability.  Fifth,  the  time  horizon  and  other  criteria 
against  which  to  predict  new  product  success  may  be  variable  or  ambiguous,  which 
exacerbates  difficulties  in  predicting  short-  and  long-term  consequences  in  analytic 
judgment.  And,  finally,  the  multitude  of  factors  that  influence  new  product  launches 
(e.g.,  the  product  itself,  its  marketing  mix,  the  marketplace,  and  the  competition)  may 
make  learning  from  new  product  experiences  quite  difficult  for  firms  and  managers. 
This  difficulty  in  learning  analytic  models  of  new  product  success  may  favor  a more 
holistic  approach  to  forecasting. 

New  product  development  is  also  an  interesting  forecasting  domain  because  of 
the  documented  evidence  of  inaccuracies  in  forecasts.  Anecdotal  evidence  suggests 
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that  many  acclaimed  experts  in  technology  and  science,  business,  and  the  arts  have 
erred  considerably  in  their  predictions  (see  Table  1).  Anecdotal  evidence  also  points 
to  the  use  of  a wide  range  of  judgmental  processes— from  relatively  analytic  to 
nonanalytic— in  new  product  forecasting.  One  example  of  remarkably  accurate 
forecasting  based  on  relatively  analytic  judgment  is  Moore’s  Law.  Gordon  E.  Moore, 
co-founder  of  Intel,  extrapolated  the  growth  in  the  number  of  transistors  on  chips  to 
predict  that  it  would  continue  to  double  every  year  for  the  next  10  years;  he  also 
reasoned  that  falling  costs  would  lead  to  an  explosion  in  chip  applications  (Business 
Week,  23  Jun  1997).  However,  analytic  judgment  may  also  lead  managers  astray. 

For  example,  Coca-Cola  may  have  relied  too  much  on  market  research  data  regarding 
taste  preferences  in  forecasting  the  success  of  New  Coke  (Hartley  1995). 

There  is  also  evidence  of  nonanalytic  judgments  in  new  product  forecasting. 
For  example,  Disney  executives  overestimated  revenue  and  profit  for  Euro  Disney  as 
a result  of  relying  on  an  analogy  to  Tokyo  Disneyland  (Hartley  1995).  Information 
technology  development  and  forecasting  also  relies  heavily  on  analogies,  most 
recently  to  ant  colonies  and  biological  organisms,  both  to  understand  future  systems 
and  to  predict  the  direction  of  their  development  (Business  Week,  23  Jun  1997).  In 
addition,  the  role  of  scenarios  in  new  product  development  and  forecasting  seems  to 
be  growing  in  acceptance,  from  Disney’s  “imagineering”  to  Motorola’s  “vision”  of  a 
wireless  future  (Hamel  & Prahalad  1994)  to  scenario  analysis  techniques  (e.g.,  Wack 
1985;  Ringland,  Todd,  & Schwartz  1998).  As  these  examples  suggest,  new  product 
forecasting  is  one  area  of  marketing  that  may  benefit  or  suffer  from  divergent 
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approaches  to  forecasting.  Certainly,  there  appears  to  be  room  for  improvement  in 
managerial  judgment.  An  estimated  45%  of  new  products  fail  after  launch  (Rosenau 
1996),  and  less  than  50%  of  new  businesses  survive  over  5 years  (Cooper,  Woo,  & 
Dunkelberg  1988).  Why  is  this  the  case,  and  how  can  these  judgmental  forecasts  be 
improved? 

Analytic  Versus  Nonanalvtic  Judgment 

The  goal  of  this  dissertation  is  to  investigate  the  effects  of  analytic  and 
nonanalytic  thought  on  judgment  within  a forecasting  context.  For  present  purposes, 
predictions  are  defined  as  judgments  of  future  events.  Events  can  refer  to  objects, 
persons,  situations,  behaviors,  or  variables  that  are  the  target  of  prediction.  These 
first-order  judgments  can  include  qualitative  predictions  of  future  event  states  (e.g., 
new  product  success/failure),  point  estimates  of  quantities  (e.g.,  new  product  sales), 
or  point  estimates  of  probabilities  (e.g.,  subjective  probability  of  success  of  a new 
product).  These  first-order  judgments  can  be  qualified  by  a second-order  judgment  of 
confidence  in  qualitative  predictions  (e.g.,  probability  that  the  prediction  is  correct)  or 
in  quantitative  predictions  (e.g.,  95%  confidence  interval  for  a quantity  or 
probability). 1 I am  primarily  interested  in  these  first-order  predictions,  although  I 
also  will  investigate  second-order  judgments  of  confidence. 

I classify  judgmental  business  forecasts  into  two  broad  categories:  analytic 
and  nonanalytic  judgments.  Analytic  judgments  reflect  people’s  efforts  to  apply 

1 The  existing  literature  is  somewhat  confusing  regarding  this  terminology.  I have  adapted  an 
approach  similar  to  Peterson  and  Pitz  (1986). 
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reasoning  and  logic  in  a relatively  objective  and  evenhanded  manner  to  arrive  at  a 
prediction.  In  addition,  people  can  use  reasoning  to  consider  evidence  (‘pros  and 
cons’)  and  causal  relationships  (based  on  a mental  model  of  causes  and  effects)  to 
arrive  at  forecasts.  For  example,  people  can  estimate  new  product  success  by 
considering  base  rates  for  product  success  or  trends  in  historical  sales  data  or  by 
identifying  and  analyzing  the  causes  of  success  or  failure  of  a new  product.  Various 
tools  have  been  proposed  in  the  literature  to  assist  analytic  judgment  (e.g.,  scenario 
analysis,  counterfactual  reasoning,  decompositional  methods  of  judgment),  and 
analytic  thought  is  also  thought  to  be  enhanced  under  certain  conditions  (e.g., 
incentives  for  accuracy,  accountability,  learning  through  feedback). 

Noncmalytic  judgments  are  comparatively  less  ‘reasonable.’  People  may 
apply  reasoning  in  a one-sided  and  subjective  manner,  drawing  on  evidence  or 
analogies.  In  addition,  people  may  use  mental  simulation  or  imagery-based 
processing  to  imagine  future  scenarios  ('mental  pictures’  of  events  leading  to  a future 
end-state  or  of  the  end-state  itself)  and  arrive  at  a prediction.  For  example,  people 
may  try  to  imagine  or  envision  the  success  of  a new  product  in  the  marketplace  (such 
as  consumers  successfully  using  the  product,  the  marketplace  conditions  under  which 
a product  would  succeed,  and/or  a scenario  of  events  leading  to  a successful  new 
product  launch).  I surmise  that  nonanalytic  thinking  (including  scenarios  and 
analogies)  may  be  relatively  common  and  even  encouraged  in  some  forecasting 
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This  distinction  between  analytic  and  nonanalytic  judgment  is  reflected  in 
prior  research.  Alba  and  Hutchinson  (1987)  discuss  analytic  and  nonanalytic 
processes  in  search,  categorization,  and  inference-making.  “Degree  of  analysis  refers 
to  the  extent  to  which  consumers  access  all  and  only  the  information  that  is  relevant 
and/or  important  for  a particular  task”  (p.417).  In  their  view,  analytic  processing 
leads  to  superior  judgment  whereas  nonanalytic  processing  is  biased  by  the  relative 
accessibility  of  information.  For  example,  similarity-based  inferences  (which  form 
the  basis  of  reasoning  by  analogy)  are  considered  nonanalytic  because  similarity  is 
based  on  salient  rather  than  causal  features.  In  contrast,  schema-based  inferences  are 
considered  analytic  inasmuch  as  they  are  driven  by  a causal  schema  and  reasoning 
about  relevant  attributes.  In  general,  analytic  processing  is  effortful  because  it  goes 
beyond  accessible  information  and  ignores  or  discounts  irrelevant  information;  hence, 
people  must  be  motivated  to  engage  in  analytic  processing.  Hutchinson  and  Alba 
(1991)  provide  evidence  that  analytic  processing  is  reduced  by  higher  memory  loads, 
incidental  (versus  intentional)  learning,  and  increased  salience  of  non-criterial 
attributes.  Nonanalytic  subjects  relied  heavily  on  salient  attributes  and  processed 
‘pseudo-analytically'  by  attending  to  a subset  of  available  information.2 

Some  researchers  propose  that  human  judgment  is  characterized  by  two 
separate  systems  of  thought  based  on  this  distinction.  In  particular,  Sloman  (1996) 

2 These  conclusions  are  reminiscent  of  those  based  on  dual  process  models  of  social  judgment, 
including  the  elaboration  likelihood  and  systematic-heuristic  models  of  persuasion  (for  a review,  see 
Eagly  & Chaiken  1993).  In  these  models,  one  judgment  route  is  argument-based  and  analytic;  the 
other  route  is  less  analytic,  influenced  by  peripheral  cues  and  heuristics.  Smith  and  DeCoster  (1999) 
and  Epstein  and  Pacini  (1999),  among  others,  provide  useful  reviews  of  evidence  for  dual  process 
models. 
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makes  a case  for  two  systems  of  reasoning.  In  the  associative  system,  processing  is 
driven  by  similarity  and  correlations,  and  judgments  are  reached  when  associations 
settle  into  a stable  state  that  maximally  satisfies  parallel  constraints  (coherence).  In 
contrast,  the  rule-based  system  “describe(s)  the  world  by  capturing  different  kinds  of 
structure,  structure  that  is  logical,  hierarchical,  and  causal-mechanical”  (Sloman 
1996,  p.6).  It  is  driven  by  rules  that  people  understand  and  apply  at  either  an  abstract 
or  pragmatic  schema  level  in  order  to  support  explanations.  ’ According  to  Sloman 
(1996),  the  two  systems  of  thought  are  complementary.  The  rule-based  system 
specializes  in  deliberate  analysis  of  relevant  features  whereas  the  associative  system 
follows  connections  without  logical  filtering.  Sloman  and  others  note  the  potential 
for  conflict  in  everyday  reasoning  and  choice  behavior.  A rule-based  structure 
provides  a more  compelling  justification  for  a response  than  does  an  associative  one, 
but  it  is  difficult  to  ignore  associative  responses. 

As  will  be  demonstrated  in  this  dissertation,  predictions  can  be  based  on 
analytic  and  nonanalytic  judgment  processes.  In  particular,  I propose  that  nonanalytic 
processes  can  have  powerful  effects  on  subsequent  predictions  and  that  the  effects  of 
nonanalytic  thinking  in  forecasting  situations  are  fairly  resistant  to  subsequent 
analytic  thought.  Thus,  the  judgmental  process  used  to  reach  a prediction  will  be 
shown  to  have  consequences  in  terms  of  susceptibility  to  errors  in  forecasting 

’ In  a similar  vein,  Allen  and  Brooks  (1991)  distinguish  between  analytic  and  nonanalytic  cognition 
based  on  whether  the  processing  is  based  on  a stored  abstraction  (prototype  or  rule)  or  an  instance  or 
exemplar.  The  key  difference  between  Allen  and  Brooks  (1991)  and  Sloman  (1996)  lies  in  their  views 
of  prototypes.  Allen  and  Brooks  view  prototypes  as  abstractions  (like  rules),  leading  to  analytic 
processing;  Sloman  views  prototypes  as  exemplars,  leading  to  nonanalytic  processing.  The  distinction 
is  not  critical  to  the  present  research. 
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accuracy.  Moreover,  nonanalytic  processes  may  inhibit  the  effectiveness  of  various 
corrective  analytic  techniques. 

Research  Propositions 

Two  types  of  nonanalytic  influences  on  forecasting  are  the  focus  of  this 
research:  scenarios  and  analogies.  To  understand  their  influence  on  judgment  (and 
the  overarching  rationale  for  this  research),  consider  what  is  likely  to  happen  if  the 
manager  is  optimistic.  I propose  that  imagining  scenarios  and  reasoning  by  analogy 
are  forms  of  elaboration  that  are  guided  by  a focal  hypothesis.  For  a plausible 
product  idea,  the  hypothesis  tends  to  be  one  of  success,  else  the  product  idea  would 
not  be  considered  seriously  by  the  manager.  Thus,  managers  tend  to  imagine 
scenarios  describing  success  and  reason  by  analogy  to  past  product  successes.  Such 
thought  is  unduly  influenced  by  associations  that  are  causally  irrelevant  to  new 
product  success  (e.g.,  mental  imagery  in  scenarios,  superficial  similarities  in 
analogies).  In  addition,  the  focal  hypothesis  inhibits  consideration  of  alternative 
hypotheses;  such  mental  sets  are  difficult  to  break  and  lead  to  confirmation  bias  and 
belief  perseverance.  As  elaboration  continues,  a schema  develops  that  supports  the 
focal  hypothesis  and  drives  subsequent  thinking.  The  schema  drives  inference 
making  and  leads  to  a well-integrated  set  of  beliefs  and  inferences.  A scenario-  or 
analogy-driven  schema  that  is  relatively  coherent  and  consistent  tends  to  be  unduly 
convincing  and  influential;  that  is,  the  whole  is  greater  than  the  sum  of  its  parts.  This 
enriched  schema  increases  the  availability  and  perceived  plausibility  of  the  focal 
hypothesis,  thereby  influencing  subsequent  judgment.  The  source  of  bias  due  to 
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nonanalytic  thinking  lies  in  the  one-sided  nature  of  scenario-driven  and  analogy- 

driven  thought.  This  one-sided  elaboration  is  a source  of  mental  contamination  that  is 

difficult  to  undo.  People  are  often  unaware  of  and  unable  to  correct  for  scenario-  or 

analogy-induced  effects  on  judgment.  Thus,  the  effects  of  nonanalytic  thinking 

persist  despite  efforts  to  debias  judgment  using  various  analytic  tools. 

This  discussion  leads  to  two  basic  propositions  underlying  this  research: 

PI . Nonanalytic  thinking  about  scenarios  and  analogies  has  powerful  effects  on 
judgment. 

P2.  The  effects  of  nonanalytic  thinking  are  resistant  to  subsequent  analytic 

thought. 

Overview 

For  ease  of  exposition,  this  dissertation  is  divided  into  two  parts.  In  the  first 
part,  I examine  scenarios  and  provide  experimental  evidence  supporting  the  basic 
propositions;  in  the  second  part,  I examine  and  provide  experimental  evidence  for 
analogies.  Each  part  of  the  paper  will  begin  with  an  overview  of  prior  literature 
related  to  nonanalytic  thinking  (scenarios  and  analogies).  This  review  will  serve  as 
background  to  support  Proposition  1 while  at  the  same  time  demonstrating  the  paucity 
of  evidence  for  Proposition  2. 

Each  set  of  studies  is  designed  to  investigate  the  effects  of  nonanalytic 
thinking  in  a new  product  forecasting  context.  The  majority  of  studies  are  conducted 
in  a controlled  laboratory  environment  using  experimental  research  techniques.  The 
laboratory  studies  are  designed  to  reflect  (simplified)  new  product  development 
settings.  In  a typical  study,  subjects  play  the  role  of  a marketing  manager  asked  to 
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forecast  new  product  success  and  sales,  given  a description  of  a new  product  concept 
(e.g.,  electric  cars,  electronic  books,  Internet  shopping).  Subjects  are  instructed  to 
engage  in  nonanalytic  thought  (e.g.,  imagining  scenarios,  reasoning  by  analogy)  prior 
to  forecasting  with  the  aid  of  various  analytic  or  debiasing  tools  (e.g.,  counterfactual 
reasoning,  accountability,  decomposition,  counter-scenarios,  counter-analogies).  The 
effectiveness  of  these  analytic  tools  is  the  focus  of  the  investigation.  Thus,  each  study 
is  designed  to  provide  evidence  for  specific  hypotheses  related  to  both  research 
propositions. 


Table  1 . 

Judgmental  forecasting  errors. 


“There  is  no  reason  for  any  individual  to  have  a computer  in  their  home.” 
(Ken  Olson,  President  of  Digital  Equipment  Corporation,  1977) 

“With  over  50  foreign  cars  already  on  sale  here,  the  Japanese  auto  industry 
isn’t  likely  to  carve  out  a big  slice  of  the  U.S.  market  for  itself.”  (Business 

Week,  2 Aug  68) 

“My  imagination  refuses  to  see  any  sort  of  submarine  doing  anything  but 
suffocate  its  crew  and  founder  at  sea.”  (H.G.  Wells,  1902) 

“In. . . Wuthering  Heights. . .all  the  faults  of  Jane  Eyre  are  magnified  a 
thousand  fold,  and  the  only  consolation  which  we  have  in  reflecting  upon 
[it]  is  that  [it]  will  never  be  generally  read.”  (James  Lorimer,  North  British 

Review,  Aug  1 849) 

“We  don’t  like  their  sound.  Groups  of  guitars  are  on  the  way  out.”  (Decca 
Recording  Company  executive,  turning  down  the  Beatles,  1 962) 

“That’s  an  amazing  invention,  but  who  would  ever  want  to  use  one  of 
them?”  (Rutherford  B.  Hayes,  U.S.  President,  1876,  after  trying  the 

telephone) 


Source:  Cerf  and  Navasky  (1984). 


CHAPTER  2 
SCENARIOS 

The  first  part  of  this  dissertation  focuses  on  scenarios.  For  present  purposes,  a 
scenario  is  defined  as  a story  that  describes  how  events  unfold  over  time  and  includes 
vivid  details  that  create  a mental  picture  or  vision  of  the  future.  Scenario  generation 
appears  to  be  an  exercise  of  the  imagination  as  people  form  a mental  picture  or  vision 
of  the  future.  Concrete  details  typically  included  in  a scenario  make  it  seem  more 
plausible  and  convincing.  Such  details  may  be  based  on  associations  and  mental 
imagery  that  are  not  necessarily  causally  relevant  to  the  future.  Scenarios  describe 
the  future  by  linking  these  elements  together  in  story-like  form. 

Prior  research  is  consistent  with  this  view  of  scenarios.  My  definition  is  very 
close  to  that  of  Russo  and  Schoemaker  (1992),  who  describe  scenarios  as  “script-like 
narratives  that  paint  in  vivid  detail  how  the  future  might  unfold  in  one  or  another 
direction”  (p.  1 3).  Kahneman  and  Tversky  (1982)  propose  that  people  run  mental 
simulations  to  construct  scenarios  or  stories  in  order  to  make  predictions  or  judgments 
under  uncertainty.  “A  good  scenario  is  one  that  bridges  the  gap  between  the  initial 
state  and  the  target  event  by  a series  of  intermediate  events,  with  a general  downhill 
trend  and  no  significant  uphill  moves  along  the  way”  (p.207).  According  to  the 
authors’  simulation  heuristic,  the  ease  of  scenario  construction  and  the  “goodness”  of 
scenarios  influence  subsequent  judgment.  Thuring  and  Jungermann  (1986)  also 
discuss  scenarios  in  terms  of  mental  simulation.  Scenarios  can  differ  in  the  degree 
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of  emphasis  placed  on  the  causal  sequence  of  events  versus  the  end-state  or  outcome 
(sometimes  called  chain  versus  snapshot  scenarios).  Other  researchers  describe 
scenarios  as  “flowing  narratives”  that  trace  the  progression  of  events  (Schnaars  & 
Topol  1987,  p.405),  stories  about  the  future  that  are  “both  causal  and  concrete”  (Kuhn 
& Sniezek  1 996,  p.2),  “focused  descriptions  of  fundamentally  different  futures 
presented  in  coherent  script-like  or  narrative  fashion”  (Schoemaker  1993,  p.  1 95),  or 
“a  narrative  description  of  a consistent  set  of  factors  which  define  in  a probabilistic 
sense  alternative  sets  of  future  business  conditions”  (Huss  1988,  p.377).  While  most 
researchers  seem  to  basically  agree  about  the  scenario  construct,  their 
operationalizations  do  differ  (as  I will  later  discuss). 

I have  previously  alluded  to  several  unique  characteristics  of  scenarios.  First, 
scenarios  include  aspects  of  imagination  and  explanation.  Second,  scenarios  describe 
the  future  using  a ‘story’  schema  that  links  beliefs  and  inferences  together  in  support 
of  a focal  hypothesis.  These  characteristics  of  scenarios  may  contribute  to  their 
powerful  effects  on  judgment,  and  there  is  some  evidence  to  support  the  importance 
of  these  characteristics  of  scenarios. 

Imagination  versus  Explanation 

As  the  previous  discussion  suggests,  scenarios  seem  to  include  aspects  of  both 
imagination  and  explanation.  While  there  is  little  prior  work  on  scenarios  per  se, 
researchers  have  investigated  these  component  processes.  The  main  thrust  of  this 
prior  research  is  that  imagination  and  one-sided  explanation  bias  judgment. 
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Imagination 

In  a typical  imagination  study,  subjects  are  instructed  to  read  or  listen  to 
presented  scenarios  that  describe  a chain  or  sequences  of  events  (e.g.,  Carroll  1978; 
Gregory,  Cialdini,  & Carpenter  1982;  Gregory,  Burroughs,  & Ainslie  1985)  or  to 
draw  specific  scenarios  leading  to  an  outcome  (Anderson  & Godfrey  1987).  For 
example,  imagining  a presidential  election  outcome  increased  its  likelihood  estimate, 
regardless  of  whether  subjects  generated  reasons  (Carroll  1 978).  Imagining  affects 
both  outcome  probabilities  and  attitudes  indirectly  related  to  the  imagined  scenario 
(Gregory  et  al.  1985).  Imagining  can  also  affect  behavior.  For  example,  people  who 
imagined  having  CATV  were  more  likely  to  predict  that  they  would  want  CATV  and 
to  later  subscribe  to  CATV  (Gregory  et  al.  1982).  The  target  and  specificity  of  the 
presented  script  also  affect  the  impact  of  the  imagined  scenario  on  judgment 
(Anderson  & Godfrey  1987).  In  cases  in  which  subjects  generate  their  own  scenarios, 
the  ease  or  difficulty  of  imagining  mediates  effects  on  judgment.  For  example, 
subjects  who  imagined  a disease  with  easy-to-imagine  symptoms  provided  higher 
likelihood  judgments  for  contracting  the  disease  than  subjects  who  imagined  difficult 
symptoms  (Sherman,  Cialdini,  Schwartzman,  & Reynolds  1985). 

Explanation 

Most  prior  research  has  operationalized  one-sided  explanation  with 
instructions  to  read  or  list  reasons  (or  arguments  or  causes)  favoring  one  side  of  an 
issue.  For  example,  explaining  the  occurrence  of  an  event  increased  the  judged 
likelihood  of  the  event  (e.g.,  a patient's  suicide),  even  when  subjects  knew  in  advance 
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that  the  event  was  hypothetical  or  when  the  event  was  discredited  afterwards  (Ross, 
Lepper,  Strack,  & Steinmetz  1977).  Similarly,  explanation  increased  the  judged 
likelihood  of  a social  theory  (e.g.,  the  relationship  between  risk-taking  and 
firefighters),  even  after  the  initial  evidence  was  discredited  (Anderson,  Lepper,  & 
Ross  1980).  Explaining  a personal  event  (e.g.,  success  or  failure  in  an  anagram  task) 
influenced  both  expectations  and  actual  behavior  (Sherman,  Skov,  Hervitz,  & Stock 
1 98 1 ),  a kind  of  self-fulfilling  prophecy  (Sherman  1 980).  All  of  these  studies 
provide  evidence  that  generating  reasons  influences  judgment  and  that  one-sided 
explanation  can  lead  to  persistent  biased  judgment  (called  “belief  perseverance”).  In 
these  studies,  the  judgmental  bias  is  typically  attributed  to  the  increased  availability 
of  information  in  memory  supporting  the  target  event  following  the  explanation  task 
(the  so-called  availability  bias;  Tversky  & Kahneman  1973). 

Because  of  its  nonanalytic  nature,  imagination  (and  scenarios)  may  be  more 
susceptible  to  salient  information,  mental  imagery  and  other  associations  than 
explanation.  Levi  and  Pryor  (1987)  compared  imagination  and  explanation  by 
distinguishing  between  “imagery  of  the  event  itself’  (outcome  imagery)  and  “reasons 
or  causes  for  the  event”.  Subjects  were  presented  with  and  instructed  to  imagine 
scenarios  (for  the  Reagan/Mondale  debate  winner)  containing  both  outcome  imagery 
and  reasons  or  outcome  imagery  alone.  Imagining  scenarios  with  both  outcome 
imagery  and  reasons  influenced  predictions;  imagining  scenarios  with  outcome 
imagery  alone  had  no  effect.  Unfortunately,  the  design  of  the  study  lacked  the  control 
groups  necessary  to  ensure  that  the  scenarios  themselves  were  equally  persuasive 
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(without  imagining).  The  authors  did  find  that  the  effect  of  imagining  the  outcome 
was  still  significant  after  controlling  for  self-generated  reasons.  However,  this  result 
is  difficult  to  interpret.  It  can  suggest  either  a true  effect  of  outcome  imagery  in 
conjunction  with  reasons  or  an  imperfect  measure  of  self-generated  reasons  in 
cognitive  responses.  The  authors  also  speculate  that  outcome  imagery  alone  could 
serve  as  a vividness  manipulation  and  influence  judgment  under  other  circumstances. 
Some  correlational  evidence  for  this  speculation  can  be  found  in  a study  of  risk 
assessment.  Personal  risk  ratings  for  events  (e.g.,  lightning,  tornado  flood  disaster) 
were  higher  for  groups  with  prior  disaster  experience.  Imagery  ratings  mediated  both 
events;  memory  clarity  mediated  lightning  only  (Greening,  Dollinger,  & Pitz  1996). 
Although  this  evidence  is  fairly  weak,  other  areas  of  research  have  found  vividness 
effects  in  judgment  (e.g.,  Pham  1994;  Reyes,  Thompson,  & Bower  1980;  but  see 
Taylor  & Thompson  1982). 

Based  on  these  findings  from  the  imagination  and  one-sided  explanation 
literature,  it  would  be  unsurprising  if  scenarios  were  found  to  bias  judgment.  Indeed, 
this  notion  is  a basic  premise  of  my  research  propositions.  However,  the  distinction 
between  imagination  and  explanation  may  be  important  when  analytic  debiasing  tools 
are  examined.  As  I discuss  in  more  detail  later,  research  on  one-sided  explanation 
suggests  that  counterexplanation  is  effective  at  debiasing  judgment.  On  the  other 
hand,  I am  unaware  of  any  findings  in  the  imagination  literature  that  address  whether 
imagination-induced  biases  can  be  eliminated  by  analytic  thinking.  If  imagination- 
induced  biases  are  difficult  to  eliminate  (which  will  be  investigated  in  Pilot  Study  1), 
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then  the  effectiveness  of  analytic  reasoning  at  eliminating  scenario-induced  biases  is 
an  open  question. 

Scenarios  versus  Reasons 

The  storytelling  nature  of  scenario  generation  may  also  be  an  important 
characteristic  that  distinguishes  scenarios  from  reasons.  Scenarios  describe  the  future 
using  a story  schema  that  links  events,  beliefs,  and  inferences  together  in  support  of  a 
focal  hypothesis.  Evidence  drawn  from  disparate  sources  suggests  that  events  in 
sequence  may  be  judged  in  a biased  fashion,  thereby  enhancing  a scenario’s 
perceived  plausibility  and  perhaps  increasing  its  resistance  to  debiasing. 

Conjunctive  Fallacy 

When  events  are  linked,  the  conjunction  is  judged  more  likely  than  its 
constituent  events,  contrary  to  the  laws  of  probability.  One  explanation  for  the 
conjunctive  fallacy  is  that  people  tend  to  assess  probability  via  availability  and 
representativeness  (Tversky  & Kahneman  1983).  Indeed,  Sloman  (1996)  uses  the 
conjunctive  fallacy  as  evidence  of  the  co-existence  of  analytic  and  nonanalytic 
systems  of  reasoning.  Scenarios  that  provide  a causal  or  temporal  sequence  to  events 
may  be  susceptible  to  this  nonanalytic  judgmental  bias. 

Story  Construction 

In  addition,  some  researchers  propose  that  people  spontaneously  evaluate 
evidence  through  “story  construction,”  attempting  to  link  events  together  into  a causal 
sequence  or  scenario  that  explains  the  outcome.  Evidence  for  spontaneous  story 
construction  and  its  influence  on  judgment  has  been  found  with  novices  and  experts 
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(Ricchiute  1998).  For  example,  a shift  in  evidence  order  changed  jury  decisions  in 
the  direction  of  the  more  easily  constructed  story  (Pennington  & Flastie  1986,  1988). 
Moreover,  evidence  organized  as  a story  leads  to  stronger  guilt  and  confidence 
judgments  than  evidence  organized  by  issue  or  reason  (motive,  character, 
opportunity)  in  a jury  context  (Pennington  & Hastie  1992).  Interestingly,  providing 
explicit  story  inference  information  (for  acquittal  versus  conviction)  had  a greater 
impact  on  global  evaluations  (based  on  story  construction)  than  on  evaluations  of 
evidence  items,  thereby  supporting  the  notion  that  a story  is  greater  than  the  sum  of 
its  arguments.  In  a similar  vein,  Teigen  (1994)  proposes  that  the  plausibility  of 
“fictional  events”  is  a function  of  completeness,  coherence,  causal  elements, 
familiarity/acceptability  (easily  assimilated  to  existing  schemas),  explicitness  of 
premises  (from  which  consequences  follow  naturally),  and  suggestiveness  (gives  rise 
to  inferences  because  it  “rings  true”). 

A scenario  or  story  elaborates  and  links  reasons  together  in  an  integrated 
causal  schema  that  explains  an  event  or  outcome.  As  previous  evidence  suggests, 
stories  appear  to  be  more  powerful  than  reasons;  hence,  scenarios  may  bias  judgment. 
Moreover,  the  distinction  between  scenarios  and  reasons  may  be  important  when 
analytic  debiasing  tools  are  examined.  Wilson  and  Brekke  (1994)  argue  that 
correcting  for  “mental  contamination”  depends  upon  the  direction  and  magnitude  of  a 
bias,  as  well  as  motivation  and  ability  to  correct  the  bias.  Their  framework  implies 
that  it  may  be  easier  to  correct  for  bias  due  to  the  generation  of  reasons  than  scenarios 
because  the  former  may  be  encoded  discretely  whereas  the  latter  are  encoded 
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integratively  in  an  elaborate  schema  that  is  difficult  to  undo.  If  elaboration  of 
scenarios  creates  biases  that  are  difficult  to  eliminate  with  evenhanded  reasons  (which 
will  be  investigated  in  Pilot  Study  2),  then  the  effectiveness  of  other  analytic 
techniques  for  eliminating  scenario-induced  biases  merits  further  investigation. 

Debiasing  Scenarios  with  Analytic  Thought 

Common  sense  and  prior  research  suggests  that  analytic  thinking  improves 
judgment.  Various  analytic  debiasing  tools  have  been  proposed  in  the  literature  to 
improve  judgment.  One  tool  consists  of  evenhanded  generation  of  reasons,  the 
traditional  form  of  analytic  thinking.  If  the  source  of  bias  is  the  increased  availability 
of  one-sided  evidence,  then  any  technique  that  increases  the  availability  of  evidence 
supporting  the  other  side  should  improve  judgment.  Such  techniques  might 
collectively  be  termed  “counterfactual  reasoning.”  Three  basic  strategies-presenting 
evenhanded  arguments,  generating  evenhanded  arguments,  and  generating  opposing 
arguments  (or  counterexplanation)-can  be  used  to  make  salient  both  sides  of  an 
argument.  The  basic  premise  of  analytic  thinking  is  that  evenhanded  consideration  of 
the  evidence  improves  judgment. 

Prior  research  clearly  indicates  that  counterexplanation  reduces  explanation- 
induced  biases.  For  example,  subjects  are  better  calibrated  if  they  list  reasons  why 
their  answers  to  almanac  questions  might  be  wrong  (Koriat  et  al.  1980).  Similarly, 
generating  con  reasons  improved  accuracy  in  predicting  positive  personal  events 
(e.g.,  MBA’s  starting  salary  estimates)  when  the  event  base  rates  were  low  to 
moderate  (Hoch  1985).  In  these  studies,  generating  supportive  reasons  had  no  effect 


20 


on  judgment,  suggesting  that  a confirmation  bias  may  occur  spontaneously.  These 
results  are  also  consistent  with  work  by  Wilson  and  LaFleur  (1995),  who  phrased 
questions  to  prompt  spontaneous  generation  of  one  or  the  other  side  of  the  argument, 
thereby  altering  the  direction  of  bias.  Mahajan  (1992)  replicated  this  effect  in  a 
marketing  context.  These  studies  constitute  evidence  that  counterexplanation,  one 
form  of  counterfactual  reasoning,  improves  judgment  after  spontaneous  explanation. 
A key  study  by  Hoch  (1984)  investigated  delay  and  order  effects  in  deliberate  pro  and 
con  argument  generation  for  various  target  events  (e.g.,  OPEC  embargo,  buying  a 
VCR).  Without  a delay,  net  reasons  were  biased  toward  initial  reasons  generated, 
which  Hoch  attributes  to  output  interference.  With  a delay  between  argument 
generation  tasks,  there  was  no  difference  in  net  reasons  generated  and  people 
achieved  evenhandedness.  Reasons  generation  influenced  judgment,  as  expected  by 
an  explanation  bias.  That  is,  predictions  showed  a primacy  effect  with  no  delay  and  a 
recency  effect  with  a delay.  Overall,  Hoch’s  work  suggests  that  a delay  between  pro 
and  con  argument  generation  will  improve  the  effectiveness  of  counterexplanation  in 
improving  judgment. 

It  is  important  to  note,  however,  that  these  results  only  address  the 
effectiveness  of  counterexplanation  in  eliminating  explanation- induced  biases.  It 
does  not  provide  any  direct  evidence  that  counterfactual  reasoning  can  eliminate 
imagination-  or  scenario-induced  biases.  Indeed,  1 have  previously  argued  that  the 
unique  characteristics  of  scenarios  may  increase  their  resistance  to  debiasing  with 
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analytic  reasoning.  Thus,  it  may  be  dangerous  to  extrapolate  these  findings  to 
scenario  contexts.  My  research  is  designed  to  address  this  limitation. 

Of  course,  common  sense  suggests  that  generating  evenhanded  reasons  or 
considering  evenhanded  evidence  should  improve  judgment  that  is  biased  by  one- 
sided elaboration.  However,  if  counterfactual  reasoning  is  differentially  effective  at 
eliminating  bias  depending  on  its  source,  then  the  source  of  the  bias  merits  further 
attention.  In  other  words,  nonanalytic  thinking  via  scenarios  may  have  consequences 
for  the  effectiveness  of  subsequent  analytic  thinking.  To  examine  this  possibility,  two 
pilot  studies  were  conducted.  The  first  pilot  study  explores  differences  in  the 
resistance  of  imagination-  and  explanation-induced  biases  to  an  evenhanded  reason- 
generation  task.  The  second  pilot  study  extends  this  investigation  of  debiasing  via 
evenhanded  reasons  to  scenarios  themselves.  If  imagination  and  scenario  effects 
prove  difficult  to  eliminate  with  evenhanded  reasoning  in  the  pilot  studies,  then  the 
effectiveness  of  other  analytic  tools  merit  further  investigation. 


CHAPTER  3 

PILOT  STUDY  1 : IMAGINATION  AND  EXPLANATION 


As  I have  argued  previously,  elements  of  both  imagination  and  explanation 
constitute  scenarios.  In  addition,  prior  research  has  demonstrated  the  effectiveness  of 
counterfactual  reasoning  following  explanation  only.  Thus,  it  may  be  unwise  to 
generalize  this  debiasing  effect.  If  imagination  resists  debiasing  via  evenhanded 
reasoning,  then  scenario-induced  biases  merit  further  investigation.  If  not,  then  the 
distinction  between  imagination,  scenarios,  and  explanation  may  be  uninteresting 
from  a debiasing  standpoint.  In  other  words,  this  study  serves  as  a pilot  to  determine 
whether  debiasing  scenarios  merits  further  investigation. 

This  exploratory  study  investigates  the  roles  of  imagination  and  one-sided 
explanation  in  probability  judgment  and  choice  among  two  alternative  new  product 
ideas  or  investment  opportunities.  Some  subjects  were  asked  to  engage  in 
explanation  and  others  were  asked  to  engage  in  imagination.  A second  factor  in  this 
study  can  be  viewed  as  a manipulation  of  the  strength  of  the  subsequent  analytic 
mindset  induced  at  choice.  In  the  explicit  reasoning  condition,  subjects  were  first 
asked  to  list  the  relative  merits  of  the  two  alternatives  before  making  their  choice.  In 
the  other  condition,  subjects  were  simply  presented  with  the  alternative  new  product 
ideas  and  asked  to  make  a choice.  Explicitly  listing  reasons  before  choice  may 
increase  the  amount  of  analytic  thinking  as  well  as  the  salience  and  importance  of 
these  reasons  in  the  decision  process.  Doing  so  could  reduce  the  impact  of 
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imagination  on  judgment  and  choice,  although  there  does  not  appear  to  be  any  reason 
why  it  should  differentially  reduce  bias  due  to  one-sided  explanation  versus 
imagination. 

Most  prior  studies  of  imagination  and  explanation  have  not  used  a time  delay 
(but  see  Gregory  et  al.  1982,  1985)  or  a choice  context,  relying  instead  on  probability 
and  attitudinal  judgments.  A choice  between  two  alternatives  may  place  managers  in 
an  analytic  mindset,  in  which  generating  reasons  for  the  alternatives  is  the  most  likely 
response.  In  addition,  a delay  leads  to  forgetting  and  reduces  the  accessibility  of  one- 
sided evidence,  thereby  reducing  output  interference  effects  (in  which  initially 
accessible  information  inhibits  the  retrieval  of  other  information)  and  potentially 
improving  the  evenhandedness  of  reasoning.  These  conditions,  then,  strengthen 
analytic  reasoning  prior  to  judgment  and  provide  a relatively  conservative  test  of 
differences  in  the  persistence  of  imagination  and  explanation  effects. 

Method 

Subjects  and  Design 

Subjects  were  undergraduate  students  who  participated  for  extra  credit  in  an 
introductory  marketing  class.  Each  subject  was  randomly  assigned  to  one  of  four 
cells  in  a 2 (Type  of  elaboration:  imagine  versus  explain)  x 2 (Explicit  reasoning 
versus  not)  between-subjects  design.  A total  of  52  subjects  participated,  with  the 
number  in  each  cell  ranging  from  11  to  15. 
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Materials  and  Procedure 


All  experimental  materials  took  the  form  of  surveys  distributed  to  each 

subject.  The  procedure  took  place  over  two  sessions,  with  a one-week  delay  between 

sessions.  In  the  first  session,  the  survey  provided  a short  description  of  a new  product 

(the  electric  car),  and  asked  subjects  to  elaborate  in  one  of  two  ways.  In  the  imagine 

condition,  subjects  were  instructed  as  follows: 

Please  take  a few  moments  to  imagine  what  it  would  be  like  if  this  product 
existed  and  was  widely  adopted  in  the  marketplace.  For  example,  imagine 
what  it  would  be  like  to  use  this  product  and/or  what  our  world  would  be 
like  if  most  cars  were  electric.  Form  a mental  picture  of  yourself  in  this 
future  world  where  electric  cars  are  in  widespread  use.  Then,  write  down 
your  vision  in  the  space  below. 

In  the  explain  condition,  subjects  were  instructed  as  follows: 

Please  take  a few  moments  to  think  of  reasons  why  this  product  might 
succeed  in  the  marketplace.  For  example,  what  are  some  of  the  reasons 
why  consumers  would  buy  this  product?  Write  down  these  reasons  in  the 
space  below. 

Subjects  were  given  a fixed  amount  of  time  to  elaborate  before  working  through  the 
rest  of  the  materials  at  their  own  pace.  Subjects  then  rated  their  attitude  toward 
electric  cars,  likelihood  of  purchase  if  available,  and  prior  knowledge.  After  a delay 
of  one  week,  subjects  then  returned  to  the  laboratory  and  were  presented  with  a 
second  survey.  This  survey  asked  subjects  to  play  the  role  of  a senior  manager  at 
Royal  Dutch/Shell  Corporation  and  to  consider  two  possible  investment 
opportunities:  investment  in  a battery  manufacturing  firm  (for  electric  cars)  or 
investment  in  a photovoltaic  cell  manufacturing  firm  (for  solar-based  home 
electricity).  After  reading  a description  of  the  investment  opportunities,  half  of  the 
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subjects  (in  the  explicit  reasoning  condition)  were  asked  to  write  down  their  thoughts 
regarding  the  relative  merits  of  the  two  alternatives  while  the  remaining  subjects 
proceeded  directly  to  additional  dependent  measures.  These  measures  took  the  form 
of  several  questions  that  asked  subjects  to  recommend  an  investment  opportunity,  rate 
their  confidence  regarding  that  recommendation  (between  50%  and  100%),  and  rate 
the  percentage  chance  of  success  of  each  opportunity  (between  0 and  1 00%)  over  the 
next  5 years,  1 0 years,  20  years,  and  beyond  20  years.  A final  thought-listing 
question  asked  subjects  for  the  rationale  behind  their  investment  choice. 

Results  and  Discussion 

CATMOD  and  LOGIT  analyses  revealed  no  statistically  significant  difference 
in  the  effect  of  the  initial  type  of  elaboration  (imagine  or  explain)  on  choice,  although 
the  difference  in  choice  is  in  the  direction  predicted  (see  Table  2).  ANOVA  revealed 
a main  effect  of  choice  on  confidence  (Fi>44=5.75,  p<021).  Subjects  who  chose  the 
imagined  or  explained  alternative  were  more  confident  than  subjects  who  chose  the 
other  alternative.  However,  the  expected  interaction  between  the  alternative  chosen 
and  the  type  of  elaboration  was  not  significant  (F<1).  Imagining  did  not  lead  to 
greater  confidence  than  explaining  when  the  imagined/explained  alternative  was 
chosen.  No  significant  effects  for  the  explicit  reasoning  factor  were  found  in  these 
analyses. 

Subjects  were  also  asked  to  rate  the  percentage  chance  of  success  of  each 
opportunity  (between  0 and  1 00%)  over  the  next  5 years,  1 0 years,  20  years,  and 
beyond  20  years.  ANOVAs  with  success  estimation  time  period  as  a within-subject 
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factor  and  type  of  elaboration  and  explicit  reasoning  as  between-subjects  factors  were 
conducted  for  each  investment  opportunity.  First,  consider  the  predictions  for  the 
electric  car.  As  one  would  expect,  estimates  of  success  increase  with  time  period 
(Fij43=269.17,  p=.0001).  This  linear  trend  is  qualified  by  an  interaction  with  type  of 
elaboration  (F);43=7.35,  p=.0096);  similarly,  the  cubic  time  trend  is  significant 
(F 1 4.3=6. 1 1,  p=.0175)  and  qualified  by  an  interaction  with  type  of  elaboration 
(Fi,43=7.69,  p=.008).  As  a graph  of  the  means  for  each  time  period  indicates,  the 
slope  for  predictions  is  greater  in  the  imagine  condition  than  in  the  explain  condition 
(see  Table  3 and  Figure  1).  A similar  analysis  of  the  predictions  for  the  alternative 
investment  opportunity,  solar  home  electricity,  indicated  that  all  effects  were  non- 
significant, as  expected.4  These  results  suggest  that  imagining  an  alternative  leads  to 
more  optimistic  predictions  regarding  success  of  that  alternative  over  time.  It  is 
interesting  to  note  that  estimates  of  future  success  appear  to  be  higher  for  5 and  10 
years  after  explaining,  and  higher  for  20  years  and  beyond  after  imagining.  A 
univariate  follow-up  analysis  of  predictions  for  each  time  period  revealed  a 
significant  effect  of  type  of  elaboration  for  beyond  20  years  only  (Fii43=4.05, 


4 A MANOVA  of  both  sets  of  predictions,  with  investment  opportunity  and  estimation  time  period  as 
within-subject  factors  and  type  of  elaboration  and  explicit  reasoning  as  between-subjects  factors, 
revealed  the  following  significant  effects:  investment  opportunity  (F,  43=3 .8 1 9,  p=.057),  time 
(Fi,43=450.07,  p=.0001),  time  X type  of  elaboration  (F , 43=5. 1 3,  p=.029),  time3  X type  of  elaboration 
(Fl,43=3  .41,  p— .072),  investment  opportunity  X type  of  elaboration  X time  (F|  43=3 .51,  p=,068)  and 
investment  opportunity  X type  of  elaboration  X time  ’ (F,  43=7. 1 8,  p=.010).  These  latter  two 
interactions  with  investment  opportunity  indicate  that  there  are  significant  differences  in  the  effect  of 
type  of  elaboration  on  predictions  for  the  imagined/explained  investment  opportunity  and  the 
alternative  opportunity,  as  one  would  expect.  For  expository  clarity,  the  results  of  the  two  separate 
ANOVAs  for  each  investment  opportunity  are  reported  in  the  main  text. 
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p=.05 1 ).'  This  suggests  that  explanation  may  have  a greater  effect  on  short-term 
judgments  and  imagination  may  have  a greater  effect  on  long-term  judgments, 
perhaps  because  subjects  may  be  more  inclined  to  rely  on  nonanalytic  judgments  than 
rational  analysis  given  the  uncertainty  inherent  in  the  long-term  judgment  task. 

In  none  of  these  analyses  was  there  a significant  effect  for  the  explicit 
reasoning  factor.  These  findings  are  notable  because  they  indicate  that  the  effects  of 
imagining  endure  despite  attempts  to  induce  an  analytic  mindset  with  or  without 
explicit  reasoning  at  the  time  of  choice.  On  the  other  hand,  null  effects  must  be 
interpreted  with  caution. 

Overall,  these  results  provide  partial  support  for  the  hypotheses.  Although  the 
imagined  alternative  was  not  chosen  significantly  more  often  nor  with  more 
confidence  than  the  explained  alternative,  there  was  a significant  effect  on  subjective 
probability  judgments  concerning  future  success  of  the  alternative.  The  most 
important  finding  is  that  imagining  the  alternative  led  to  more  optimistic  predictions 
of  future  success  than  explaining  the  alternative.  Even  after  a one-week  delay  and  an 
explicit  evenhanded  reasoning  task,  subjects  were  influenced  by  a prior  imagination/ 
explanation  task  that  lasted  only  a few  minutes.  Inasmuch  as  imagination  is  a 
characteristic  element  of  scenarios,  then  scenarios  may  also  have  enduring  effects  that 
resist  debiasing  with  evenhanded  reasoning. 


3 Explicit  reasoning  is  marginally  significant  for  predictions  for  the  next  5 years  (F,  43=3.28,  p=.077): 
subjects  who  listed  reasons  for  choice  provided  lower  estimates  for  the  first  time  period  only. 
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Table  2. 


Pilot  study  1 : choice  and  confidence  after  imagining/explaining  success  of  a target 

alternative. 


Type  of 
Elaboration 

Reasoning 
at  Choice 

CHOICE 
(of  target 
alternative) 

CONFIDENCE 
(in  choice  of 
target  alternative 

CONFIDENCE 
(in  choice  of  non- 
target alternative 

Imagine 

Not  explicit 

6/11 

84.09 

71.25 

73.3% 

(8.01) 

(11.09) 

Imagine 

Explicit 

9/15 

83.33 

78.33 

60.0% 

(13.46) 

(12.11) 

Explain 

Not  explicit 

6/11 

77.50 

69.00 

54.5% 

(12.55) 

(14.75) 

Explain 

Explicit 

6/11 

80.83 

75.00 

54.5% 

(8.61) 

(12.75) 

Table  3. 

Pilot  study  1 : probability  of  success  over  time  after  imagining/explaining  success  of 

a target  alternative. 

Time  Period: 

over  next  5 
years 

over  next  1 0 
years 

over  next  20 
years 

beyond  20 
years 

Imagine 

13.70 

33.89 

67.93 

88.00 

(13.34) 

(19.58) 

(23.86) 

(16.33) 

Explain 

20.25 

40.65 

58.60 

75.70 

(20.60) 

(22.08) 

(21.95) 

(25.15) 

Note:  For  purposes  of  expository  clarity,  the  results  are  not  displayed  as  a function  of 


explicit  reasoning. 


Probability  of  Success 
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5 years  10  years  20  years  beyond  20 

Time  Period 


— Explain 


Imagine 


Figure  1. 

Pilot  study  1 : probability  of  success  over  time  after  imagining/explaining  success  of 

a target  alternative. 


CHAPTER  4 

PILOT  STUDY  2:  SCENARIOS  AND  ELABORATION 


In  the  second  pilot  study,  my  investigation  shifts  from  imagination  to  formal 
scenarios,  the  focus  of  the  first  part  of  this  dissertation.  The  objective  of  this  pilot 
study  is  twofold.  First,  it  is  designed  to  demonstrate  the  resistance  of  scenarios  to 
debiasing  with  evenhanded  reasoning.  This  objective  follows  logically  from  the 
previous  study,  which  demonstrated  the  resistance  of  imagination  (a  unique 
characteristic  of  scenarios)  to  debiasing  with  evenhanded  reasons.  If  evenhanded 
reasons  fail  to  eliminate  scenario  effects,  then  other  analytic  tools  merit  investigation. 
Second,  this  study  is  designed  to  demonstrate  that  elaboration  of  a scenario  is  a 
source  of  persistence.  Prior  research  suggests  that  stories  are  stronger  than  reasons, 
and  I have  argued  previously  that  scenario  generation  leads  to  the  construction  of  an 
elaborate  story  schema  to  support  the  focal  hypothesis,  i.e.,  success  or  failure.  Thus, 
further  elaboration  of  this  scenario  should  bolster  support  for  the  hypothesis  and 
polarize  judgment.  Insofar  as  people  tend  to  think  about  and  elaborate  on  a scenario, 
then  this  could  account  for  the  persistence  of  scenario-induced  biases  over  time. 

Support  for  the  biasing  effect  of  additional  thought  is  drawn  from  the 
polarization  literature,  which  suggests  that  directed  (or  one-sided)  thought  leads  to 
polarized  judgment.  In  this  stream  of  research,  Tesser  and  Leone  (1977)  initially 
demonstrated  that  people  make  more  polarized  judgments  for  attitude  objects  after 
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schema-driven  thought.  This  finding  contrasted  with  later  work  by  Linville  and 
others,  who  demonstrated  that  attitude  polarization  depends  on  whether  the  schema  is 
simple  or  complex  (Linville  & Jones  1980;  Linville  1982).  To  reconcile  these 
findings,  Millar  and  Tesser  (1986)  proposed  that  commitment  and  cognitive  structure 
account  for  the  contradictory  findings  of  Tesser  and  Linville.  When  people  are 
committed  to  initial  evaluations,  thought  leads  to  polarization  in  the  presence  of  more 
complex  schemas.  When  uncommitted,  thought  leads  to  polarization  when  driven  by 
a simple  versus  complex  schema.  In  these  studies,  commitment  and  the  cognitive 
structure  can  be  viewed  as  influencing  the  direction  of  thought.  Similarly,  adopting  a 
focal  hypothesis  and  elaborating  on  this  hypothesis  (i.e.,  constructing  a story  or 
scenario)  should  also  induce  directional  thought  and  lead  to  polarized  judgment. 

In  this  pilot  study,  subjects  were  instructed  to  imagine  either  new  product 
success  or  failure.  Some  subjects  were  also  given  the  opportunity  to  carry  out  further 
elaboration  of  the  initial  scenario.  As  in  the  previous  study,  I predict  that  the 
scenario-induced  bias  will  endure  despite  subsequent  instructions  to  generate 
evenhanded  pro  and  con  reasons.  Moreover,  additional  thought  or  elaboration  of  a 
scenario  will  enhance  its  effect  on  judgment. 

By  contrasting  success  and  failure  scenarios,  the  effects  of  imagination  can  be 
generalized  to  both  success  and  failure  scenarios.  In  this  study,  subjects  were  also 
given  a lengthy  and  evenhanded  new  product  description  (consistent  with  prior 
research  on  story  construction),  in  contrast  to  the  short  description  provided  in  the 
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previous  study.  Because  the  results  in  the  previous  study  were  somewhat  sensitive  to 
the  type  ot  dependent  variable,  I included  additional  dependent  measures  related  to 
new  product  success.  Whereas  the  first  pilot  study  manipulated  the  strength  of 
analytic  thinking  in  an  attempt  to  demonstrate  differences  in  debiasing,  this  study 
manipulated  the  strength  of  the  imagined  scenario  in  an  attempt  to  demonstrate 
differences  in  induced  bias. 

Method 

Subjects  and  Design 

Subjects  were  undergraduate  students  who  participated  for  extra  credit  in  an 
introductory  marketing  class.  Each  subject  was  randomly  assigned  to  one  of  four 
cells  in  a 2 (Type  of  scenario:  success  versus  failure)  x 2 (Amount  of  elaboration: 
low  versus  high)  between-subjects  design.  A total  of  48  subjects  participated,  with 
the  number  in  each  cell  ranging  from  10  to  13. 

Materials  and  Procedure 

All  experimental  materials  took  the  form  of  surveys  distributed  to  each 
subject.  Subjects  worked  through  the  materials  at  their  own  pace.  Subjects  played 
the  role  of  a new  product  manager  and  read  a three-page  description  of  the  current 
status  of  a new  product  idea  (electronic  books).  The  description  was  deliberately 
designed  to  be  neutral  and  to  provide  subjects  with  mixed  information  regarding  the 
industry  players  and  customers  for  the  new  product.  Subjects  indicated  their  initial 


b To  increase  the  generalizability  of  results  across  studies  and  to  simplify  this  study’s  design,  this  study 
did  not  incorporate  a delay  or  choice  and  used  a different  target  new  product  in  the  stimulus  materials. 
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opinion  of  the  new  product  idea  and  rated  the  product  information.  They  then  were 
instructed  to  take  at  least  5-10  minutes  to  write  either  a success  or  failure  scenario  as 


follows: 

Failure  Scenario:  You  have  just  read  a report  that  gives  you  some 
background  information  about  the  current  status  of  Electronic  Books. 

Now  we  would  like  you  to  take  some  time  to  think  about  a worst-case 
scenario  in  which  E-Books  will  not  succeed  in  the  future.  (A  scenario  is  a 
story  that  describes  how  events  unfold  over  time  and  includes  vivid  details 
that  create  a mental  picture  or  vision  of  the  future.)  Write  a scenario  that 
describes  a story  of  failure  for  E-Books. 

Success  Scenario:  You  have  just  read  a report  that  gives  you  some 
background  information  about  the  current  status  of  Electronic  Books. 

Now  we  would  like  you  to  take  some  time  to  think  about  a best-case 
scenario  in  which  E-Books  will  succeed  in  the  future.  (A  scenario  is  a 
story  that  describes  how  events  unfold  over  time  and  includes  vivid  details 
that  create  a mental  picture  or  vision  of  the  future.)  Write  a scenario  that 
describes  a success  story  for  E-Books. 

After  writing  this  scenario,  subjects  indicated  their  opinion  again  and  rated 

various  aspects  of  their  written  scenario.  These  materials  were  then  removed  and 

subjects  completed  a short  filler  task.  Subjects  in  the  high  elaboration  conditions 

were  then  asked  to  elaborate  on  their  scenario  as  follows: 

In  scenario  planning,  people  usually  find  that  it  is  worthwhile  to  spend 
additional  time  working  on  their  scenario.  Please  take  a few  more  minutes 
to  think  about  the  scenario  that  you  wrote  about  Electronic  Books.  Try  to 
recall  it  in  as  much  detail  as  you  can.  Feel  free  to  fill  in  details  and 
elaborate  on  events  in  your  scenario.  Work  on  developing  your  scenario 
more  fully.  Write  down  this  scenario  in  the  space  below.  (If  you  need 
more  space,  please  use  the  other  side  of  the  page.) 

Subjects  in  the  low  elaboration  condition  proceeded  directly  to  a second  filler 


task. 
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After  a second  short  filler  task,  all  subjects  generated  evenhanded  reasons  for 
the  success  and  failure  of  E-Books.  Subjects  then  indicated  their  opinion  again  and 
answered  various  other  questions  regarding  the  future  of  E-Books. 

Results  and  Discussion 

This  study  predicted  a two-way  interaction  between  the  type  of  scenario  and 
the  amount  of  elaboration.  Additional  elaboration  of  the  scenario  was  expected  to 
exacerbate  the  difference  between  judgments  following  a success  versus  failure 
scenario.  The  overall  pattern  of  results  across  the  dependent  measures  is  consistent 
with  this  hypothesis. 

Consider  first  several  direct  measures  of  attitudes  toward  E-Books.  Subjects 
were  asked  to  rate  their  agreement  with  several  statements  concerning  the  success  of 
E-Books:  “E-Books  are  destined  to  succeed  in  the  marketplace,”  “E-Book  sales  will 
take  off  and  grow  rapidly,”  “People  will  be  reluctant  to  try  this  new  technology,”  and 
“After  some  initial  interest,  E-Book  sales  are  likely  to  stall  in  the  marketplace.”  (The 
final  two  statements  were  reverse-coded.)  A MANOVA  revealed  a significant 
interaction  of  the  type  of  scenario  and  the  amount  of  elaboration  (F!>44=5.36,  p=  025), 
as  well  as  a main  effect  of  the  type  of  scenario  (FM4=4.93,  p=. 032).  As  the  means  in 
Table  4 indicate,  subjects  were  less  optimistic  about  the  E-Book's  success  if  they 
generated  and  elaborated  on  a failure  versus  success  scenario.  Follow-up  analyses  of 
the  high  elaboration  groups  reveals  a significant  effect  of  type  of  scenario 
(Fi>44-10.82,  p=.002);  otherwise,  there  was  no  scenario  effect  (F<1).  Thus, 
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elaboration  polarized  judgments;  as  predicted,  generating  evenhanded  reasons  was 
insufficient  to  eliminate  the  bias  if  the  scenario  was  given  additional  elaboration. 

This  pattern  of  results  is  also  found  on  several  other  less  direct  judgment 
measures  (see  also  Table  4).  For  example,  subjects  were  asked  to  predict  attitudes 
toward  Internet  shopping  for  a number  of  different  groups  (e.g.,  adults,  teens,  seniors, 
schools,  publishing  companies,  retailers,  etc.).  A MANOVA  of  these  predictions 
revealed  a significant  two-way  interaction  of  type  of  scenario  and  amount  of 
elaboration  (F i 44=3 .3  8,  p=. 073),  as  well  as  a main  effect  of  type  of  scenario 
(F],44=7.58,  p=.009).  In  planned  contrasts,  the  effect  of  type  of  scenario  was 
significant  under  high  elaboration  (Fi,44=1  1.09,  p=.002);  otherwise,  type  of  scenario 
had  no  effect  (F<1).  In  other  words,  predictions  were  more  (less)  favorable  toward 
Internet  shopping  when  subjects  generated  and  elaborated  on  a success  (failure) 
scenario.  A similar  pattern  of  means  is  evident  in  predictions  for  the  attractiveness 
in  electronic  form  of  various  reading  materials  (e.g.,  best-sellers,  classic  literature, 
children’s  books,  school  text  books,  business  books,  popular  magazines,  professional 
magazines  and  newsletters,  etc.)  but  does  not  reach  significance.  Personal  attitudes 
were  also  affected  by  a two-way  interaction  of  the  type  of  scenario  and  amount  of 
elaboration  (Fi  44=5. 1 4,  p=.028).  Elaborating  about  a success  (failure)  scenario  led  to 
more  (less)  positive  personal  opinions  about  E-Books  (F144=6.46,  p=. 015);  this 


For  brevity’s  sake,  these  results  are  not  reported  separately  in  tables  in  this  chapter.  Results  are 
available  from  the  author  upon  request. 
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difference  was  not  significant  in  the  low  elaboration  groups  (F<1).  Thus,  results  so 
far  provide  strong  support  for  the  hypothesis. 

Some  additional  dependent  measures  were  taken  repeatedly  and  their  analysis 
deserves  special  attention.  First,  subjects’  initial  attitude  toward  E-Books  (after 
reading  the  background  information)  did  not  differ  across  groups  (all  effects  were 
non-significant).  Second,  the  initial  scenario  generation  task  had  an  immediate  effect: 
attitudes  were  more  (less)  favorable  (Fi  41=22.31,  p=.0001)  and  predictions  were  more 
(less)  optimistic  (F1j41=10.85,  p=.002)  after  a success  (failure)  scenario.  Third, 
attitudes  were  not  consistently  different  after  the  reason  generation  task.  Attitudes 
were  slightly  more  favorable  for  subjects  who  generated  a success,  versus  failure, 
scenario  (Fi,44=3.43,  p=.071)  but  no  more  optimistic;  contrary  to  the  hypothesis, 
additional  elaboration  had  no  effect  (although  the  means  in  Table  5 are  in  the 
expected  direction).  Finally,  this  same  pattern  is  evident  when  these  attitudes  were 
measured  again  later  in  the  survey  (all  F<1).  Flowever,  at  this  point,  an  additional 
attitude  measure  that  asked  subjects  to  assess  the  rate  of  growth  of  E-Books  did  reveal 
the  hypothesized  two-way  interaction  (FM3=5.09,  p=.029;  see  Figure  2).  Subjects 
assessed  a more  rapid  growth  rate  after  generating  and  elaborating  about  a success, 
versus  failure,  scenario  (Fii44=7.21,  p=.010).  The  growth  rate  did  not  differ  for 
subjects  who  did  not  carry  out  additional  elaboration  (F<1).  This  significant 
interaction  effect  for  growth  rate  reflects  the  same  pattern  of  means  found  on  the  non- 
repeated  measures  discussed  previously. 
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Subjects  were  also  asked  to  predict  the  percentage  of  Americans  that  will 
adopt  E-Books  over  time  (see  Table  6).  A MANOVA  of  these  estimates  revealed  a 
significant  three-way  interaction  of  type  of  scenario,  amount  of  elaboration  and 
estimation  period  (cubic  polynomial;  Fi  44=6.51,  p=.014).  Follow-up  analyses 
revealed  marginal  interactions  of  type  of  scenario  and  estimation  period  under  both 
high  and  low  elaboration  (cubic  polynomial;  high:  Fi>44=2.95,  p=.093;  low: 

Fj  44=3.56,  p=. 066).  Unexpectedly,  the  pattern  of  means  is  in  the  opposite  direction: 
low  elaboration  produced  larger  scenario  differences  than  high  elaboration.  Subjects 
were  also  asked  to  estimate  market  share  for  electronic  book  materials  over  time  (see 
also  Table  6).  A MANOVA  reveals  an  interaction  of  the  amount  of  elaboration  with 
estimation  period  (cubic  polynomial;  F)i44=4.76,  p=. 035),  also  qualified  by  an 
interaction  with  the  type  of  scenario  (Fi ,44=4.67,  p=.036).  In  planned  contrasts  in 
both  elaboration  conditions,  the  interaction  of  type  of  scenario  and  estimation  period 
failed  to  reach  significance  (cubic  polynomial;  high:  F|,44=2.56,  p=.l  17;  low: 

Fi  ,44=2.14,  p=.  151),  probably  due  to  low  power.  As  the  pattern  of  means  for  the 
significant  two-way  interaction  suggests,  estimates  were  more  (less)  optimistic 
following  generation  and  elaboration  of  a success  (failure)  scenario.  This  latter  result 
is  supportive. 

Overall,  these  results  indicate  that  elaboration  about  scenarios  leads  to 
polarized  judgments.  In  addition,  generating  evenhanded  reasons  was  insufficient  to 
eliminate  a scenario-induced  bias  after  additional  elaboration.  However,  generating 
pro  and  con  reasons  was  effective  at  debiasing  judgment  when  scenario  elaboration 
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was  limited.  From  a practical  standpoint,  the  latter  result  can  be  taken  as  evidence 
that  the  reason  generation  task  was  an  effective  manipulation  (countering  the  usual 
criticisms  of  null  effects  and  demand).  From  a theoretical  standpoint,  this  study 
provides  a boundary  condition  on  scenario  effects  by  suggesting  that  the  power  of 
scenarios  lies  in  their  elaboration.  Elaborate  scenarios  resist  debiasing  via 
evenhanded  reasoning;  the  question  then  turns  to  whether  other  analytic  tools  are 
effective. 


Table  4. 

Pilot  study  2:  judgments  after  generating  and  elaborating  a scenario  for 
failure/success  of  electronic  books. 


Type  of 
Scenario 

Amount 

of 

Elabo- 

ration 

N 

Agreement 

with 

Success 

Statements 

(Averaged) 

Predictions 
of  Others’ 
Attitudes 
(Averaged) 

Attractive- 
ness of 
Reading 
Materials 
(Averaged) 

Personal 

Attitudes 

(Averaged) 

Failure 

High 

13 

2.40  (0.71) 

3.72  (0.67) 

4.45  (1.09) 

4.19(1.74) 

Failure 

Low 

13 

2.61  (0.35) 

3.98  (0.63) 

4.60  (0.96) 

4.90(1.28) 

Success 

High 

12 

3.15  (0.42) 

4.47  (0.55) 

4.97  (0.86) 

5.56  (1.02) 

Success 

Low 

10 

2.60  (0.71) 

4.13  (0.27) 

4.53  (0.60) 

4.50(1.18) 
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Figure  2. 

Pilot  study  2:  growth  rate  of  electronic  books  after  scenario  elaboration  and 
evenhanded  reasons  generation. 


Table  5. 


Pilot  study  2:  attitude  ratings  measured  after  scenario  and  pro/con  generation  and 
______  then  repeated  after  additional  questioning. 


Type  of 

Amount  of 

Attitude  After  Pro/Con  Generation 

Scenario 

Elaboration 

N 

Favorability 

Success 

Growth  Rate 

Failure 

High 

13 

3.38  (1.61) 

3.62  (1.80) 

2.77(1.59) 

Failure 

Low 

13 

4.00  (1.00) 

4.15  (1.28) 

3.23  (1.54) 

Success 

High 

12 

4.50(1.51) 

4.50(1.17) 

3.08  (1.08) 

Success 

Low 

10 

4.30  (0.95) 

4.20  (1.03) 

2.70  (0.82) 

Attitude  Repeated  After  Additional  Questioning 

N 

Favorability 

Success 

Growth  Rate 

Failure 

High 

13 

3.54(1.71) 

3.92  (2.06) 

2.46(1.56) 

Failure 

Low 

13 

4.15  (1.28) 

4.23  (1.54) 

3.08(1.12) 

Success 

High 

11 

4.18(1.54) 

4.73  (1.35) 

3.82  (1.08) 

Success 

Low 

10 

4.30(1.16) 

4.30(1.16) 

2.80(1.03) 

Note:  Initial  attitudes  and  post-scenario  attitudes  are  not  reported  due  to  space 
limitations. 
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Table  6. 

Pilot  study  2:  adoption  and  market  share  estimates  over  time  for  electronic  books. 


Type  of 

Amount  of 

N 

% of  Americans  Adopting  Ebooks  over  Time 

Scenario 

Elaboration 

2005 

2010 

2020 

Beyond  2020 

Failure 

High 

13 

13.69 

25.77 

44.38 

57.23 

(20.37) 

(23.26) 

(27.05) 

(28.21) 

Failure 

Low 

13 

15.85 

28.85 

43.08 

53.85 

(18.37) 

(20.79) 

(28.74) 

(38.31) 

Success 

High 

12 

17.75 

30.92 

45.00 

62.08 

(20.15) 

(30.21) 

(31.41) 

(30.49) 

Success 

Low 

10 

20.69 

33.62 

57.37 

70.62 

(18.09) 

(20.40) 

(27.29) 

(32.41) 

% Market  Share  for  Electronic  Books  over  Time 

Failure 

High 

13 

20.38 

25.54 

38.77 

47.38 

(28.32) 

(18.16) 

(27.11) 

(32.52) 

Failure 

Low 

13 

23.55 

31.77 

46.92 

56.15 

(26.35) 

(25.07) 

(33.86) 

(39.06) 

Success 

High 

12 

19.25 

35.58 

49.84 

66.76 

(17.83) 

(29.61) 

(31.57) 

(31.57) 

Success 

Low 

10 

18.20 

29.40 

55.30 

66.30 

(17.29) 

(19.13) 

(28.28) 

(30.41) 

CHAPTER  5 

STUDY  1 : SCENARIO  ANALYSIS 
The  pilot  studies  suggest  that  imagination  and  elaboration  contribute  to 
scenario  effects  on  judgment.  These  studies  also  provide  evidence  that  one  debiasing 
tool,  evenhanded  reason  generation,  failed  to  eliminate  the  effects  of  scenarios  on 
judgment.  In  light  of  these  persistent  effects  of  scenarios,  Study  1 strengthens 
debiasing  efforts  by  using  two  additional  analytic  tools,  counterfactual  reasoning  and 
presented  evidence,  in  an  effort  to  eliminate  scenario  effects  on  judgment. 
Counterfactual  reasoning  heightens  the  availability  of  opposing  information  and 
therefore  can  be  viewed  as  a stronger  debiasing  tool  compared  to  evenhanded  reasons 
(as  in  the  pilot  studies).  From  an  availability  perspective,  presented  evidence  is  also  a 
stronger  manipulation  of  analytic  thinking  compared  to  generated  evidence  because  it 
eliminates  memory-mediated  effects  that  reduce  the  accessibility  of  opposing 
arguments  after  a one-sided  elaboration  task.  In  this  study,  both  of  these  analytic 
tools  are  used  in  an  attempt  to  improve  judgment.  Thus,  this  study  provides  the 
strongest  test  yet  of  scenario-induced  biases  and  the  effectiveness  of  analytic  tools  in 
debiasing  judgment. 

This  study  also  offers  direct  evidence  of  “bias.”  Heretofore,  I have 
demonstrated  scenario  “effects”  on  judgment,  which  allows  me  to  conclude  that 
judgments  are  flawed  (inasmuch  as  differential  predictions  cannot  all  be  accurate).  In 
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the  absence  of  actual  outcome  data  (which  is  not  available  for  these  new  products), 
the  accuracy  of  predictions  cannot  be  ascertained  nor  can  “bias”  (which  exists  when 
predictions  systematically  differ  from  actual  outcomes).  However,  judgment  can  be 
called  biased  if  it  is  path  dependent.  That  is,  processing  order  in  judgment  is  causally 
unrelated  to  actual  outcomes,  thereby  inducing  systematic  effects  on  judgment  that 
constitute  bias. 

In  addition,  this  study  is  designed  to  investigate  scenario  analysis,  a 
management  tool  for  improving  forecasting.  In  scenario  analysis,  managers  first 
generate  several  alternative  future  scenarios  and  then  forecast  the  future.  Proponents 
of  scenario  analysis  argue  that  scenarios  help  managers  deal  with  an  uncertain  future 
by  simplifying  it  into  a limited  number  of  alternatives  (Wack  1985).  The  main 
benefits  of  scenario  analysis  from  a debiasing  perspective  are  said  to  include  a 
reduction  in  both  overconfidence  and  myopic  framing  (Schoemaker  1992,  1995). 
From  a psychological  perspective,  the  effectiveness  of  scenario  analysis  may  be 
rooted  in  two  processes: 

1 . counterfactual  reasoning,  mediated  by  availability  of  arguments  (e.g.,  Hoch 
1984) 

2.  multiple  explanation,  mediated  by  the  ease  of  simulation  of  alternative 
outcomes  (e.g.,  Hirt  & Markman  1995;  Koehler  1991,  1994). 

I have  already  discussed  prior  psychological  research  related  to  counterfactual 
reasoning.  The  effectiveness  of  counterfactual  reasoning  is  mediated  by  the 
availability  of  supporting  and  opposing  arguments  that  influence  judgment.  In 
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contrast,  research  on  multiple  alternatives  suggests  that  considering  alternative 
hypotheses  (whether  opposing  or  not)  is  sufficient  to  improve  judgment.  In  the 
discussion  that  follows,  I will  review  prior  research  in  this  area. 

Multiple  Explanation 

Hypothesis  Generation 

People  generally  experience  difficulty  in  generating  alternatives.  They  stop 
too  soon  when  generating  alternatives  (Shaklee  & Fischhoff  1982),  generate  new 
hypotheses  only  when  existing  hypotheses  seem  implausible  (Gettys  & Fisher  1979) 
and  underestimate  the  other  ungenerated  hypotheses  (Gettys,  Mehle,  & Fisher  1986). 
On  the  bright  side,  when  people  do  generate  alternative  hypotheses,  they  become 
better  calibrated  in  their  judgments.  Asking  people  to  generate  hypotheses  in  the 
other  category  reduces  (but  does  not  eliminate)  overconfidence  in  a focal  hypothesis 
(Mehle  1981).  In  general,  the  fault  tree  literature  demonstrates  that  people  do  not 
spontaneously  generate  other  hypotheses  or  alternatives  but  when  prompted  to  do  so 
are  less  overconfident.  For  example,  likelihood  estimates  for  a focal  cause  (e.g., 

Bill’s  death  due  to  smoke  inhalation)  are  reduced  by  the  number  of  other  causes 
generated  and  are  sensitive  to  the  likelihood  of  alternative  causes  (Dougherty,  Gettys, 
& Thomas  1997).  However,  generating  alternatives  appears  to  be  a difficult  task. 
Both  experts  and  novices  experience  difficulty  generating  a complete  set  of 
hypotheses  and  therefore  are  overconfident  in  their  judgments  (Fischhoff,  Slovic,  & 
Lichtenstein  1978;  Mehle  1982).  Moreover,  incentives  do  not  improve  performance 
(Gettys,  Pliske,  Manning,  & Casey  1987).  People’s  inability  to  generate  an 
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appropriate  set  of  alternatives  has  important  implications  in  a judgment  or  prediction 
context.  For  example,  Dube-Rioux  and  Russo  (1988)  found  that  managers  generated 
more  short-term  than  long-term  causes  of  restaurant  failure,  which  led  to  a greater 
under-estimation  bias  for  long-term  causes  of  failure.  In  addition,  a causal  focus 
developed  early  in  hypothesis  generation  (for  solutions  to  parking  problems) 
influences  subsequent  hypotheses  generated  (Adelman,  Gualtieri,  & Stanford  1995). 
This  result  is  consistent  with  other  research  suggesting  that  people  have  difficulty 
generating  alternatives  because  alternatives  generated  earlier  inhibit  subsequent  idea 
generation  by  restricting  thinking.  For  example,  people  tend  to  imitate  an  exemplar 
solution  when  designing  bicycle  racks  (Jansson  & Smith  1991)  or  drawing  alien 
creatures  (Ward  1994;  Marsh,  Landau,  & Hicks  1996),  which  suggests  the  importance 
of  the  starting  point  or  initial  condition  in  any  generation  task. 

Hypothesis  Evaluation 

People  also  experience  difficulty  in  evaluating  multiple  alternatives.  They 
exhibit  confirmation  bias  in  the  search  for  evidence  (Klayman  & Ha  1989)  and  fail  to 
consider  how  evidence  supports  other  alternatives  during  testing  (Tversky  & Koehler 
1994;  Griffin  & Tversky  1992).  Moreover,  both  novices  and  experts  are  imperfect 
hypothesis  testers.  For  example,  Asare  and  Wright  (1995)  showed  that,  although 
expert  auditors  did  not  persevere  in  beliefs  and  were  not  influenced  by  nondiagnostic 
evidence,  they  did  encode  evidence  and  evaluate  hypotheses  independently  (one 
hypothesis  syndrome)  and  sometimes  resuscitated  eliminated  hypotheses.  Novices 
made  all  these  normative  errors  and  additional  substantive  mistakes  as  well.  Hence, 
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people  are  non-Bayesian  when  it  comes  to  hypothesis  evaluation  (see  Fischhoff  & 
Beyth-Marom  1983  for  a review),  which  has  important  implications  for  debiasing. 

For  example,  evidence  increasing  the  suspiciousness  of  one  suspect  in  a mystery  story 
does  not  decrease  the  suspiciousness  of  alternative  suspects  (Van  Wallendael  & 

Hastie  1990).  Similarly,  subjects  under-  (over-)  estimate  a focal  candidate’s  job 
chances  when  evidence  in  the  candidate’s  favor  is  weak  (strong),  because  they  fail  to 
consider  that  the  evidence  is  weak  (strong)  for  the  other  candidates  as  well 
(Sanbonmatsu,  Posavac,  & Stasney  1997).  In  other  words,  people  seem  to  treat 
multiple  alternatives  in  hypothesis  testing  as  independent,  which  suggests  that 
instructing  people  to  evaluate  how  evidence  supports  the  alternatives  may  be  a 
debiasing  strategy.  In  research  by  Gibson,  Sanbonmatsu,  and  Posavac  (1997), 
estimating  a focal  team’s  probability  of  winning  led  to  more  gambling,  compared  to  a 
control  group  that  provided  estimates  for  all  of  the  teams.  Here,  estimating  the 
alternatives  debiased  judgment,  perhaps  by  forcing  people  to  consider  evidence  for 
each  of  the  alternatives  rather  than  selectively  testing  the  focal  alternative.  Similarly, 
choosing  an  answer  among  alternatives  may  reduce  overconfidence  compared  to 
merely  evaluating  pre-selected  answers  to  general  knowledge  questions  and 
predictions  of  basketball  games  (Ronis  & Yates  1987;  Sniezek,  Paese,  & Switzer 
1990).  These  results  suggest  that  any  task  that  leads  people  to  consider  evidence  for 
all  of  the  alternatives,  rather  than  just  testing  the  focal  alternative,  will  improve 


judgment. 
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Multiple  Alternatives 

Hirt  and  Markman  (1995)  provide  a powerful  demonstration  of  the 
effectiveness  of  multiple  alternatives  in  debiasing  judgment.  For  example,  subjects 
who  explained  a ‘close  win’  by  team  A rated  team  A’s  chances  of  winning  as  higher. 
Predictions  were  subsequently  debiased  if  subjects  then  explained  a ‘close  win’  by  the 
opposing  team  (a  traditional  counterfactual  reasoning  task)  or  a ‘convincing  win’  by 
team  A (an  alternative  explanation  task).  Explaining  an  implausible  alternative 
outcome  (e.g.,  a low-ranked  team  wins  its  league)  failed  to  debias  judgment,  which 
suggests  a mediating  role  of  ease-of-simulation  (Hirt  & Markman  1995).  Judgment 
also  improved  after  a multiple  explanation  task  involving  positive,  negative,  and  zero 
relationships  between  risk-taking  and  firefighting  (Hirt  & Markman  1995;  see  also 
Anderson  & Sechler  1986  for  a counterexplanation  version  of  this  study).  Hirt  and 
Markman  argue  that  generating  alternative  explanations  undermines  confidence  in  the 
focal  hypothesis,  thereby  improving  judgment. 

Koehler  (1994)  proposes  that  the  consideration  of  alternatives  improves 
judgment  by  breaking  the  conditional  frame  of  reference  induced  by  the  focal 
hypothesis.  Subjects  who  generated  their  own  alternatives  (e.g.,  predicting  Academy 
Award  winners,  answering  general  knowledge  questions)  were  better  calibrated  than 
subjects  who  evaluated  presented  alternatives.  Koehler  argues  that  having  to  generate 
the  alternative  increases  the  salience  of  multiple  contenders,  thereby  improving 
judgment.  Indeed,  when  the  salience  of  alternatives  was  held  constant  by  presenting 
all  of  the  alternatives,  choosing  among  alternatives  increased  overconfidence 
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compared  to  evaluating  pre-selected  answers  (Koehler  1 994).  Koehler  argues  that  the 
act  of  choosing  increased  supportive  arguments  in  favor  of  one’s  choice,  an  effect 
opposite  that  found  in  Ronis  and  Yates  (1987)  and  Sniezek  et  al.  (1990).  It  is  difficult 
to  reconcile  these  results  with  Hirt  and  Markman’s  finding,  unless  something  beyond 
the  mere  salience  of  alternatives  is  necessary  to  break  the  inertia  of  a conditional 
frame  of  reference. 

Interestingly,  some  evidence  suggests  that  people  may  not  always  be 
appropriately  sensitive  to  multiple  alternatives,  making  it  more  difficult  to  break  this 
inertia.  For  example,  people  identified  a person  checking  window  locks  in  a story  as 
a window  installer  and  ignored  cues  concerning  a burglar  (Ross  & Murphy  1996). 
Thus,  the  mere  salience  of  alternatives  (generated  or  presented)  may  be  insufficient  to 
debias  judgment,  unless  sufficient  attention  is  drawn  to  the  alternative  perhaps  (for 
example,  via  an  explanation  task,  as  in  Hirt  & Markman  1995).  Indeed,  Hirt  and 
Markman’s  finding  is  especially  interesting  in  light  of  the  fact  that  it  was  based  on  a 
similar  study  conducted  by  Anderson  and  Sechler  (1986).  In  this  study,  predictions 
for  various  social  theories  (e.g.,  positive  versus  negative  relationship  between  fire- 
fighting and  risk-taking)  changed  in  the  direction  indicated  by  an  explanation  task. 
While  a counterexplanation  task  was  completely  effective  in  debiasing  judgment, 
exposing  subjects  to  contradictory  data  (e.g.,  zero  relationship)  did  not  completely 
debias  judgment.  In  this  case,  subjects  moderated  their  beliefs  somewhat  but  did  not 
abandon  them,  perhaps  because  people  are  reluctant  to  replace  a strong  belief  in  a 
relationship  with  a null  belief  or  because  the  null  data  was  especially  susceptible  to 
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biased  interpretation.  In  addition,  there  may  be  differences  in  the  impact  of 
arguments  that  are  presented  versus  generated. 

Nonetheless,  Hirt  and  Markman’s  finding  does  provide  important  evidence 
that  multiple  alternatives  are  sufficient  to  debias  judgment.  Although  researchers 
have  proposed  that  multiple  alternatives  debias  judgment  by  improving  the  evaluation 
of  alternatives  or  breaking  the  conditional  frame  of  reference,  it  can  be  difficult  to 
distinguish  the  proposed  mechanisms  in  practice  because  the  former  subsumes  the 
latter.  Regardless  of  the  proposed  mechanism,  research  on  multiple  explanation  has 
typically  focused  on  analytic  reasoning  tasks  and  may  not  be  safely  generalized  to 
nonanalytic  thinking. 

Scenario  Analysis 

The  premise  of  scenario  analysis  is  that  prompting  people  to  consider 
alternative  future  scenarios  will  improve  judgment.  Several  scenario  studies  support 
this  contention  and  include  some  evidence  that  multiple  scenarios  reduce 
overconfidence.  For  example,  instructing  subjects  (somewhat  heavy-handedly)  to 
stretch  their  estimate  limits  by  considering  best-case  and  worst-case  outcomes 
increased  confidence  intervals  for  duration  of  a software  project  (Connolly  & Dean 
1997).  Similarly,  subjects  prompted  to  make  multiple  construals  widened  their 
confidence  intervals  for  self  and  other  predictions  (e.g.,  money  spent  on  dinner,  time 
spent  answering  phone  survey)  compared  to  other  subjects  (Griffin,  Dunning,  & Ross 
1 990).  Evidence  from  the  control  groups  suggests  that  people  typically  generate  a 
single  construal  of  a situation  and  make  predictions  as  if  this  construal  were 
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certain/correct.  Schoemaker  (1993)  also  found  that  both  self-generated  and  presented 
scenarios  led  to  wider  confidence  ranges.  However,  scenarios  did  not  always  reduce 
overconfidence,  perhaps  because  they  were  sometimes  perceived  as  extreme  and 
therefore  implausible.  Interestingly,  the  beneficial  effects  of  scenario  analysis  were 
obtained  even  though  subjects  still  exhibited  desirability  biases,  favored  positive 
trends  and  outcomes  more  highly,  and  were  susceptible  to  the  conjunctive  fallacy. 
These  judgmental  errors  seem  to  suggest  the  fallibility  of  scenario  analysis  as  an 
analytic  tool  to  improve  judgment. 

Several  other  studies  are  only  weakly  supportive,  if  at  all,  for  scenario 
analysis.  For  example,  subjects  who  made  a series  of  predictions  based  on  graphical 
and  scenario  information  exhibited  increased  confidence  with  multiple  scenarios,  but 
accuracy  did  not  improve  correspondingly  (Schnaars  & Topol  1987).  Similarly, 

Kuhn  and  Sniezek  (1996)  found  that  confidence  in  prediction  increased  with  any  type 
of  presented  scenarios  (unidirectional,  conflicting  or  hybrid  scenarios,  compared  to  no 
scenario)  whereas  point  estimate  forecasts  and  confidence  intervals  were  unaffected. 
In  another  study  that  investigated  personal  predictions  (Byram  1997),  subjects  who 
estimated  the  time  required  to  assemble  a computer  stand  provided  “best  guess” 
predictions  that  were  closer  to  optimistic  than  pessimistic  scenario  predictions.  There 
was  no  evidence  of  order  effects  when  generating  these  multiple  predictions,  and 
generating  multiple  estimates  (optimistic,  pessimistic  and  best-guess  versus  best- 
guess  only)  did  not  improve  judgment.  This  optimism  bias  is  reminiscent  of  the 
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planning  fallacy,  in  which  people  tend  to  focus  on  plan-based  scenarios  rather  than 
considering  the  range  of  past  outcomes  (e.g.,  Buehler,  Griffin,  & Ross  1994). 

Taken  together,  these  findings  raise  questions  regarding  the  efficacy  of 
scenario  analysis.  In  these  various  studies,  the  operationalization  of  scenario  analysis 
ranged  from  fairly  elaborate  scenarios  generated  by  subjects  (e.g.,  Schoemaker  1995; 
Griffin  et  al.  1990),  to  simple  reasons  presented  to  subjects  (Schnaars  & Topol  1987; 
Kuhn  & Sniezek  1 996),  to  little  more  than  the  brief  consideration  of  the  range  of 
possible  outcomes  (Byram  1997).  Based  on  this  distinction,  only  elaborate  multiple 
scenarios  improved  judgment;  mere  salience  or  simple  consideration  of  alternatives 
was  insufficient.8  Because  proponents  of  scenario  analysis  typically  describe 
scenario  development  as  a relatively  effortful  process,  results  of  studies  that  have 
used  more  elaborate  scenarios  seem  more  relevant  tests  of  the  technique’s 
effectiveness.  Most  of  the  studies  using  elaborate  scenarios  have  been  supportive 
(Connolly  & Dean  1997;  Schoemaker  1993;  Griffin  et  al.  1990;  Hirt  & Markman 
1995),  although  some  are  not  (Schnaars  & Topol  1987;  Kuhn  & Sniezek  1996).  The 
latter,  however,  differed  in  two  aspects:  the  scenario  information  was  presented  and 
resembled  reasons  rather  than  true  scenarios.  Of  course,  other  differences  among  the 


8As  previously  discussed,  Koehler  (1994)  argues  that  mere  salience  is  sufficient  to  debias  judgment. 
Perhaps  the  difficulty  of  generating  one’s  own  hypothesis  (when  no  alternatives  are  made  salient), 
rather  than  the  salience  of  alternative  hypotheses  arising  out  of  generation,  accounted  for  reduced 
confidence  in  judgment  (consistent  with  his  experiments  1,  4,  5 and  6).  When  alternatives  were  held 
salient,  choosing  (versus  evaluating)  increased  confidence,  perhaps  via  the  increased  availability  of 
supportive  arguments  (experiments  2 and  3).  The  latter  is  also  consistent  with  Ronis  and  Yates  (1987) 
and  Sniezek  et  al.  (1990). 
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studies,  such  as  the  type  of  judgment  (e.g.,  personal  versus  impersonal  predictions), 
increase  the  difficulty  of  interpreting  and  reconciling  these  contradictory  findings. 

Perhaps  the  litmus  test  for  the  effectiveness  of  scenario  analysis  is  whether  it 
is  possible  to  generate  and  evaluate  multiple  scenarios  in  an  unbiased  manner.  Biases 
in  the  generation  and  evaluation  process  would  undermine  the  effectiveness  of 
scenario  analysis  as  an  aid  to  judgmental  forecasting.  Prior  research  has  found  order 
effects  in  generating  reasons  (e.g.,  Hoch  1984).  Prior  research  on  story  construction, 
as  well  as  results  from  the  pilot  studies,  suggests  that  the  effect  of  scenarios  may  be 
stronger  than  the  effect  of  reasons.  If  so,  order  effects  may  be  more  pronounced  for 
scenarios  than  reasons.  Once  an  elaborate  and  integrated  schema  has  been  developed 
in  support  of  an  initial  scenario,  it  may  be  difficult  to  'undo'  this  scenario  completely. 
Based  on  this  line  of  reasoning  and  my  prior  evidence,  I predict  that  scenario  analysis 
will  be  susceptible  to  order  effects.  That  is,  predictions  will  be  biased  in  the  direction 
of  the  initial  scenario  imagined  (primacy  order  effect).  Generating  counter-scenarios 
will  not  debias  judgment. 

Method 

Subjects  and  Design 

Subjects  were  undergraduate  students  who  participated  for  extra  credit  in  an 
introductory  marketing  class.  Each  subject  was  randomly  assigned  to  one  of  four 
cells  in  a 2 (Type  of  elaboration:  generate  scenarios  versus  reasons)  x 2 (Order: 
success  then  failure  versus  failure  then  success)  x 2 (Repeat  predictions:  after 
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scenarios  and  after  presented  evidence)  mixed  design.  A total  of  92  subjects 
participated,  with  the  number  in  each  cell  ranging  from  21  to  25. 

Materials  and  Procedure 

All  experimental  materials  took  the  form  of  surveys  distributed  to  each 
subject.  The  survey  asked  subjects  to  play  the  role  of  a business  executive  and  think 
about  the  future  of  Internet  shopping.  Subjects  were  given  some  background 
information  on  the  current  status  of  Internet  shopping  and  asked  to  give  an  initial 
professional  opinion.  Some  subjects  then  generated  a best-case  scenario  for  the 
success  of  Internet  shopping;  other  subjects  generated  reasons  for  the  success  of 
Internet  shopping.  Subjects  then  generated  a scenario  or  reasons  for  failure.  Order 
was  manipulated  between-subjects,  so  other  subjects  generated  reasons/scenarios  for 
failure  followed  by  success.  (The  reasons  conditions  serve  as  control  groups  in  this 
design.) 

Each  writing  task  was  timed  and  lasted  5 minutes.  In  the  success  scenario 

writing  task,  the  title  of  the  writing  task  was  “Best-Case  Scenario:  Internet  Shopping 

is  a Success.”  Subjects  were  instructed  as  follows: 

Please  think  about  a best-case  scenario  in  which  Internet  shopping  will 
succeed.  (Recall  the  characteristics  of  a scenario— it  is  a story  that  describes 
how  events  unfold  over  time  and  includes  vivid  details  that  create  a mental 
picture  or  vision  of  the  future.)  Write  a scenario  that  describes  this  success 
story  for  Internet  shopping. 

In  the  failure  scenario  writing  task,  the  title  of  the  writing  task  was  “Worst-Case 

Scenario:  Internet  Shopping  is  a Failure.”  Subjects  were  instructed  as  follows: 

Please  think  about  a worst-case  scenario  in  which  Internet  shopping  will  not 
succeed.  (Recall  the  characteristics  of  a scenario— it  is  a story  that  describes 
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how  events  unfold  over  time  and  includes  vivid  details  that  create  a mental 
picture  or  vision  of  the  future.)  Write  a scenario  that  describes  this  story  of 
failure  for  Internet  shopping. 

Similarly,  in  the  writing  task  for  success  reasons,  the  writing  task  was  titled  “Best- 

Case:  Internet  Shopping  is  a Success”  and  subjects  were  instructed  as  follows: 

Please  think  about  reasons  why  Internet  shopping  will  succeed.  List  as  many 
reasons  as  you  can  to  support  the  success  of  Internet  shopping. 

In  the  writing  task  for  failure  reasons,  the  writing  task  was  titled  “Worst-Case: 

Internet  Shopping  is  a Failure”  and  subjects  were  instructed  as  follows: 

Please  think  about  reasons  why  Internet  shopping  will  not  succeed.  List  as 
many  reasons  as  you  can  to  support  the  failure  of  Internet  shopping. 

Subjects  then  responded  to  a series  of  questions  concerning  the  future  of 

Internet  shopping,  including  a final  opinion,  market  share  predictions  over  time, 

likelihood  and  confidence  measures,  recommendations  for  launch  of  on-line  sales  in 

various  product/service  categories,  and  predictions  of  attitudes  of  various  population 

groups.  Subjects  also  provided  some  background  information  (e.g.,  prior 

knowledge).  Subjects  were  then  presented  with  a set  of  pro  and  con  reasons 

representing  “another  opinion  about  the  future  of  Internet  shopping”  (which  may  be 

“helpful  or  not  in  forming  your  professional  opinion”). 

Results  and  Discussion 

The  study  predicted  a two-way  interaction  between  type  of  elaboration 
(scenario  versus  reasons)  and  order.  Judgments  were  expected  to  be  more  biased 
toward  an  initial  scenario  than  toward  initial  reasons  generated.  Results  across  all  of 
the  dependent  measures  are  consistent  with  this  hypothesis. 
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First,  consider  subjects’  direct  ratings  of  the  likelihood  of  success  and  failure 
for  Internet  shopping  (see  Table  7).  Subjects  were  asked  to  indicate  whether  success 
or  failure  best  describes  the  future  of  Internet  shopping  and  also  to  rate  the  likelihood 
of  the  reasons/scenarios  that  they  wrote  describing  success  and  failure.  A CATMOD 
analysis  on  the  binary  choice  between  success  and  failure  indicates  a significant  two- 
way  interaction  of  type  and  order  of  elaboration.  Subjects  predicted  failure  outcomes 
more  often  when  their  initial  scenario  supported  failure  or  their  second  set  of  reasons 
supported  failure  (x,2=4.40,  p=.036).  In  addition,  analyses  of  likelihood  ratings  also 
indicated  a significant  two-way  interaction  of  type  and  order  of  elaboration 
(Fi,87=8.13,  p=.005).  Likelihood  ratings  reflect  a primacy  effect  for  scenarios  and  a 
recency  effect  for  reasons.  The  recency  effect  in  the  reasons  control  groups  rules  out 
fatigue  and  demand  as  alternative  explanations,  whereas  the  primacy  effect  for 
scenarios  supports  the  study’s  hypothesis. 

This  pattern  of  results  is  also  found  on  several  other  less  direct  judgment 
measures  displayed  in  Table  8.  For  example,  subjects  were  asked  to  recommend 
whether  a firm  should  launch  on-line  sales  in  a variety  of  product/service  categories 
(e.g.,  lawyer  services,  housing,  furniture,  hardware,  clothing,  etc.).  The  number  of 
launch  recommendations  appears  to  be  driven  by  the  two-way  interaction  of  type  and 
order  of  elaboration  (Fi,8g=3.86,  p=053).9  Subjects  were  also  asked  to  predict 
attitudes  toward  Internet  shopping  for  a number  of  different  groups  (e.g.,  adults, 

9 A MANOVA  of  subjects’  confidence  across  all  of  these  recommendations  also  revealed  a significant 
two-way  interaction  of  type  and  order  of  elaboration  (Fii86=3.40,  p=.069). 
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teens,  seniors,  manufacturers,  retailers,  catalogers,  etc.)  as  well  as  their  personal 
attitude.  A MANOVA  of  these  predictions  also  revealed  a significant  two-way 
interaction  of  type  and  order  of  elaboration  (Fi,87=6.48,  p=.013).  Overall,  predictions 
were  more  favorable  toward  Internet  shopping  when  subjects  generated  a success 
scenario  initially  or  when  subjects  generated  reasons  for  success  more  recently.10 

Some  additional  repeated  dependent  measures  were  taken  and  their  analysis 
deserves  special  attention.  First,  consider  the  attitude  measures  that  were  taken  at 
three  points  in  time:  initially  (prior  to  elaboration),  after  the  ordered  elaboration  task, 
and  after  presenting  pro  and  con  reasons  to  subjects.  These  attitude  judgments  can  be 
used  to  construct  two  attitude  change  measures  to  examine  whether  changes  in 
attitude  are  induced  by  the  ordered  elaboration  tasks  and  by  presenting  pro  and  con 
information.  A MANOVA  of  both  attitude  change  measures  indicates  a significant 
two-way  interaction  of  type  of  elaboration  and  order  (Fi  ,88=4.97,  p-028).*  11  As  the 
results  in  Table  9 and  Figure  3 indicate,  attitude  changes  appear  to  depend  upon  both 
the  type  of  elaboration  and  the  order  of  elaboration.  For  reasons,  there  is  a recency 
effect  of  the  second  set  of  reasons  generated:  attitudes  become  more  negative  over 
time  in  the  pro-con  order  and  more  positive  over  time  in  the  con-pro  order.  In 
contrast,  for  scenarios,  there  is  a primacy  effect  of  the  initial  scenario:  attitudes 
become  more  positive  over  time  in  the  best-case-worst-case  order  and  more  negative 

10  For  brevity’s  sake,  these  results  are  not  reported  separately  in  tables  in  this  chapter.  Results  are 
available  from  the  author  upon  request. 

11  In  addition,  there  is  a significant  interaction  of  type  of  elaboration  and  the  two  attitude  change 
measures  (F,  88=6.21,  p=.015),  but  this  must  be  interpreted  in  light  of  the  two-way  interaction  of  type 
of  elaboration  and  order. 
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over  time  in  the  worst-case-best-case  order.  Although  the  interaction  with  repetition 
is  not  significant,  it  does  appear  at  least  directionally  from  the  means  that  the  two- 
way  interaction  becomes  stronger  over  time  (Fi,g8=l  .95,  p=0.167).  If  this  were  the 
case,  then  it  would  suggest  that  presenting  relatively  evenhanded  reasons  can  actually 
exacerbate  the  order-driven  bias,  rather  than  reducing  it.  In  any  case,  there  is  no 
indication  that  presenting  pro  and  con  reasons  was  effective  at  reducing  the  bias  in 
judgment. 

Second,  consider  predictions  for  market  share,  which  were  repeated  both  after 
the  ordered  elaboration  task  and  after  presenting  pro  and  con  information  (see  Table 
10).  Subjects  were  asked  to  estimate  the  percentage  of  all  retail  sales  that  will  take 
place  on  the  Internet  in  2003,  2008,  2013,  and  2018  and  beyond.  A MANOVA  of 
these  estimates  indicates  a significant  four-way  interaction  of  type  of  elaboration, 
order  of  elaboration,  estimation  period  (cubic  polynomial),  and  repeated  judgment 
(Fi.83-4.63,  p=. 034).  Follow-up  analysis  indicates  that  the  three-way  interaction 
(type  X order  X period3)  was  not  significant  (although  the  means  are  directional)  for 
market  share  judgments  following  initial  elaboration  but  was  significant  following 
presentation  of  reasons  (Fi.85-6.33,  p=.014).  As  Figure  4 indicates,  the  slopes  for 
market  share  over  time  are  higher  when  the  initial  scenario  described  success  (versus 
failure)  and  when  the  second  set  of  reasons  described  success  (versus  failure). 

An  alternative  analysis  of  this  data  examined  the  change  in  market  share 
predictions  with  repeat  judgment  (see  Table  10).  A MANOVA  indicated  a marginal 
two-way  interaction  of  type  and  order  of  elaboration  (F 1 85=3 .05,  p=.085),  qualified 
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by  an  interaction  with  success  estimation  period  (F3,83=2. 1 95,  p=.095).  Follow-up 
univariate  analyses  indicate  that  the  two-way  interaction  has  an  effect  on  changes  in 
market  share  for  2003  (Fiig5=2.99,  p=.087),  2008  (Fi,g5=4.00,  p=.049),  2013  (F<1) 
and  2018  and  beyond  (Fi, 85=3. 58,  p=. 062).  In  other  words,  although  initial  market 
share  judgments  were  not  significantly  affected  by  type  and  order  of  elaboration, 
repeated  judgments  after  presenting  reasons  did  reveal  a significant  bias  that  was 
driven  by  the  primary  scenario  or  recent  reasons.  Figure  5 is  a graph  of  the  average 
change  across  time  periods.  The  results  suggest  that  presenting  pro  and  con  evidence 
may  exacerbate  rather  than  correct  bias  in  judgment  and  are  also  consistent  with  the 
direction  of  results  for  the  attitude  change  measure. 

Although  this  latter  finding  is  not  conclusive,  it  merits  additional  research. 
Even  the  failure  of  presented  evidence  to  debias  judgment  is  surprising.  Common 
sense  suggests  that  people  will  base  judgments  on  presented  evidence;  indeed,  this  is 
a basic  premise  of  a great  deal  of  research.  On  the  one  hand,  a null  effect  could 
merely  indicate  a weak  manipulation.  For  example,  the  presented  evidence  may  have 
been  perceived  as  weak  or  unconvincing,  although  this  is  not  indicated  by  the 
ratings.13  On  the  other  hand,  some  prior  research  suggests  that  presented  evidence 
can  have  unintended  effects  on  judgment.  For  example,  Lord,  Ross,  and  Lepper 


12  Such  was  the  case  in  a study  by  Anderson  and  Sechler  (1985),  who  found  that  subjects  moderated 
their  beliefs  in  a social  theory  somewhat  but  did  not  abandon  them  when  exposed  to  contradictory  data 
(e.g.,  zero  relationship  between  risk-taking  and  fire-fighting).  Their  study  is  particular  interesting 
because  it  forms  the  basis  for,  and  contrasts  with,  Hirt  and  Markman’s  subsequent  work  demonstrating 
that  multiple  explanation  debiases  judgment. 

13  Pro  evidence  was  rated  4.93  (1.32)  on  an  unconvincing-convincing  1-7  scale.  Con  evidence  was 
rated  4.66  (1 .48).  Ratings  did  not  vary  significantly  by  condition. 


58 


(1979)  demonstrated  biased  assimilation  of  pro/con  evidence  presented  to  subjects 
and  polarization  of  attitudes  (toward  the  death  penalty).  Similarly,  Miller, 

McHoskey,  Bane,  and  Dowd  (1993)  found  that  biased  assimilation  of  evidence  and 
attitude  polarization  occurred  especially  among  people  holding  extreme,  versus 
moderate,  attitudes.  Perhaps  presented  evidence  polarizes  judgment  if  it  enables 
subjects  to  focus  on  supportive  evidence  and  ignore  or  discount  opposing  evidence.  In 
a sense,  evenhanded  evidence  contains  further  arguments  that  serve  as  grist  for  the 
mill  and  aid  people  in  constructing  explanations  to  support  and  bolster  their  initial 
attitude.  However,  Lord  et  al.’s  result  has  not  always  been  replicated  (e.g.,  Anderson 
& Sechler  1986;  for  a review,  see  Kuhn  & Lao  1996)  and  this  study’s  evidence  is  not 
strong. 

Overall,  the  results  of  this  study  provide  considerable  support  for  the  notion 
that  there  are  order  effects  in  judgment  for  scenarios  and  reasons.  First,  primacy 
order  effects  were  stronger  for  scenarios  than  reasons:  generating  an  initial  scenario 
influenced  judgment  more  than  generating  initial  reasons.  In  other  words,  scenarios 
lead  to  ‘stickier  priors’  than  reasons.  Second,  order  effects  endure  despite 
counterfactual  thinking;  hence,  judgments  remained  biased  in  the  direction  of  an 
initial  scenario  generated,  even  after  the  generation  of  a counter-scenario.  This 
susceptibility  to  order  effects  raises  questions  about  the  efficacy  of  scenario  analysis, 
which  relies  on  the  generation  of  multiple  alternative  scenarios  to  improve 
forecasting.  And,  finally,  I have  presented  some  evidence  that  subsequent  attempts  at 
debiasing  judgment  via  presentation  of  reasons  may  in  fact  exacerbate  rather  than 


eliminate  bias.  Certainly,  there  is  no  evidence  that  providing  evenhanded  facts 
eliminated  the  scenario-induced  bias  in  judgment. 
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Table  7. 


Type  of 
Elaboration 

Order 

# of  Subjects 
predicting 
success 

Likelihood 
that  IS 
succeeds 

Likelihood 
that  IS  fails 

Reasons 

Success- 

14/20 

4.76(1.51) 

3.48(2.71) 

Failure 

(70%) 

Reasons 

Failure- 

19/21 

5.48  (1.12) 

2.71  (1.38) 

Success 

(90%) 

Scenarios 

Success- 

22/25 

5.28(1.14) 

3.12(1.20) 

Failure 

(88%) 

Scenarios 

Failure- 

18/25 

4.63  (1.47) 

3.92  (1.56) 

Success 

(72%) 

Table  8. 


Study  1 : indirect  judgments  after  ordered  elaboration  for  Internet  shopping. 


Type  of 
Elaboration 

Order 

# of  Launch 
Recommendations  for 
Products/Services 

Predictions  of 
Others’  Attitudes 
(averaged) 

Reasons 

Success- 

Failure 

4.71  (2.08) 

4.28  (0.72) 

Reasons 

Failure- 

Success 

6.24(1.14) 

4.68  (0.74) 

Scenarios 

Success- 

Failure 

5.40  (1.73) 

4.63  (0.69) 

Scenarios 

Failure- 

Success 

5.56  (1.56) 

4.15  (0.96) 
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Table  9. 


Study  1 : attitudes  toward  Internet  shopping  before  and  after  ordered  elaboration  and 
after  presented  evidence.  


Type  of 
Elaboration 

Order 

Initial 

Attitude 

Attitude 

after 

Elabor- 

ation 

Attitude 

after 

Presented 

Evidence 

Differ- 
ence after 
Elabor- 
ation 

Differ- 
ence after 
Presented 
Evidence 

Reasons 

Success- 

4.33 

4.14 

4.29 

-0.19 

-0.05 

Failure 

(1.24) 

(1.46) 

(1.55) 

(1.25) 

(1.02) 

Reasons 

Failure- 

4.56 

4.88 

5.19 

0.31 

0.62* 

Success 

(1.29) 

(1.14) 

(1.44) 

(0.93) 

(1.20) 

Scenarios 

Success- 

4.56 

4.88 

4.80 

0.32 

0.24 

Failure 

(1.12) 

(1.05) 

(1.04) 

(0.90) 

(0.72) 

Scenarios 

Failure- 

4.72 

4.84 

4.36 

0.12 

-0.36 

Success 

(1.21) 

(1.43) 

(1.60) 

(1.42) 

(1.55) 

Note:  * indicates  change  in  attitude  is  significantly  different  from  0 (p=.017). 

However,  the  possibility  of  attitude  drift  implies  that  contrasts  to  zero  are  an 
inappropriate  analysis. 


Study  1 : change  in  attitude  after  ordered  elaboration  (A)  and  presented  evidence  (B). 


61 


Table  10. 


Study  1 : market  share  estimates  for  internet  shopping  after  ordered  elaboration  and 


after  presented  evidence. 


Type  of 
Elaboration 

Order 

Market  Share  Estimates  over  Time 
(Measured  after  Ordered  Elaboration) 
2003  2008  2013  2018+ 

Reasons 

Success 

8.27 

14.74 

22.96 

34.33 

-Failure 

(12.44) 

(18.04) 

(25.63) 

(34.03) 

Reasons 

Failure- 

6.29 

14.65 

24.19 

37.67 

Success 

(8.89) 

(14.41) 

(22.68) 

(29.84) 

Scenarios 

Success 

4.15 

12.47 

24.39 

37.67 

-Failure 

(4.67) 

(11.82) 

(20.13) 

(26.67) 

Scenarios 

Failure- 

6.52 

14.41 

25.48 

36.02 

Success 

(5.84) 

(11.94) 

(21.65) 

(28.64) 

Market  Share  Estimates  over  Time 

(Measured  after  Presented  Evidence) 

2003 

2008 

2013 

2018+ 

Reasons 

Success 

6.87 

13.29 

22.65 

31.64 

-Failure 

(11.74) 

(17.04) 

(25.76) 

(33.15) 

Reasons 

Failure- 

7.67 

15.22 

23.88 

36.26 

Success 

(10.04) 

(13.61) 

(19.54) 

(27.37) 

Scenarios 

Success 

4.03 

12.26 

21.64 

35.04 

-Failure 

(4.19) 

(11.71) 

(16.96) 

(25.10) 

Scenarios 

Failure- 

5.84 

10.44 

19.99 

29.74 

Success 

(6.34) 

(8.30) 

(18.03) 

(27.18) 

Change 

Change 

Change 

Change 

Average 

in  MS 

in  MS 

in  MS 

in  MS 

Change 

2003 

2008 

2013 

2018  + 

in  MS 

Reasons 

Success 

-1.4 

-1.4 

-0.3 

-2.7 

-1.4 

-Failure 

(5.4) 

(7.83) 

(11.3) 

(5.2) 

(6.5) 

Reasons 

Failure- 

1.4 

0.6 

-0.3 

2.2 

+0.9 

Success 

(6.8) 

(8.8) 

(11.4) 

(15.0) 

(9.8) 

Scenarios 

Success 

-0.1 

-0.2 

-2.7 

-2.6 

-1.4 

-Failure 

(1.1) 

(3.6) 

(6.9) 

(7.8) 

(3.6) 

Scenarios 

Failure- 

-0.7 

-4.0 

-5.5 

-6.3 

-4.1 

Success 

(3.1) 

(6.2)* 

(9.2)* 

(11.9)* 

(6.6)* 

Note:  * indicates  change  is  significantly  different  from  0 (p<.01).  However,  the 
possibility  of  attitude  drift  implies  that  contrasts  to  zero  are  an  inappropriate  analysis. 
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Figure  4. 

Study  1 : market  share  estimates  for  Internet  shopping  after  presented  evidence  in  the 
“ordered  reasons”  conditions  (A)  and  “ordered  scenarios”  conditions  (B). 


-• — Reasons  — ■-  Scenarios 


Figure  5. 

Study  1 : average  change  across  presented  evidence  in  market  share  estimates  for 

Internet  shopping. 


CHAPTER  6 

STUDY  2:  SCENARIOS  AND  ACCOUNTABILITY 
The  previous  studies  suggest  that  counterfactual  reasoning  is  ineffective  at 
debiasing  judgment  induced  by  an  initial  scenario.  Neither  generated  nor  presented 
evenhanded  reasons  eliminated  scenario  effects;  generated  counter-scenarios  also 
failed.  In  this  study,  I try  a different  approach  and  investigate  whether  accountability 
will  debias  judgment. 

Popular  wisdom  suggests  that  accountability  (expecting  to  have  to  justify 
one’s  opinions  to  others,  a common  feature  of  the  business  world)  is  sufficient  to 
avoid  many  judgmental  biases  demonstrated  in  a laboratory  setting.  Some  research 
suggests  that  accountability  improves  judgment  by  inducing  “more  integratively 
complex”  thinking  (i.e.,  analytic  thought).  In  a number  of  studies,  accountability  has 
been  found  to  induce  evenhanded  consideration  of  evidence  (Tetlock  1983a)  and 
reduce  judgmental  biases  (Tetlock  1985;  Tetlock  & Kim  1987;  Simonson  & Nye 
1992).  For  example,  the  order  effect  in  generating  pro  and  con  evidence  in  a court 
case  was  reduced  when  people  expected  to  justify  their  position  to  others  (Tetlock 
1983b).  When  Tetlock  and  Boettger  (1994)  showed  people  evidence  of  the  risks  and 
benefits  of  a new  drug  product,  accountability  led  to  risk  aversion  in  a launch 
decision.  However,  subsequent  research  suggests  that  accountability  is  not  a broad 
panacea  for  correcting  errors  in  judgment  (Simonson  & Nye  1992;  Tetlock  & 
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Boettger  1989).  For  example,  accountability  has  been  shown  to  exacerbate 
confirmation  bias  in  judgments  of  causal  responsibility  (Sanbonmatsu,  Akimoto,  & 
Biggs  1993).  In  an  attempt  to  reconcile  some  of  these  conflicting  findings,  Tetlock, 
Skitka,  and  Boettger  (1989)  suggest  that  accountability  demands  can  be 
accommodated  in  three  ways:  bolstering  one’s  own  position  if  already  committed  to 
it,  conforming  to  known  or  suspected  views  of  one’s  audience14,  and  thinking  in 
cognitively  complex  ways  {pre-emptive  self-criticism). 

Lerner  and  Tetlock  (1999)  have  recently  synthesized  prior  research  on 
accountability.  Accountability  is  said  to  attenuate  bias  that  “results  from  lack  of 
effort,  self-critical  awareness  of  one’s  judgment  process,  or  both,”  i.e.,  by  prompting 
pre-emptive  self-criticism  (p.263).  If  so,  one  would  expect  accountability  to  enhance 
analytic  thinking  by  motivating  people  to  consider-the-opposite  or  multiple 
alternatives  in  expectation  of  having  to  justify  their  views.  In  contrast,  accountability 
is  said  to  amplify  bias  that  “results  from  naive  use  of  normatively  (but  not  obviously) 
irrelevant  cues”  (p.264).  I have  previously  argued  that  scenarios  incorporate 
imagination  and  story-telling,  nonanalytic  features  or  irrelevant  cues  that  are  self- 
generated and  bias  judgment.  If  so,  accountability  may  exacerbate  bias  if  it  motivates 
people  to  elaborate  more  extensively  using  a nonanalytic  thought  process  such  as 
scenarios.  Therefore,  I propose  that  accountability  will  exacerbate  overconfidence 


14  In  a field  study,  Brown  (1999)  finds  that  accountability  leads  managers  to  ‘policy  bolster’  by 
conforming  to  socially  acceptable  decision  strategies.  When  asked  to  account  for  their  views, 
analytically-oriented  professionals  increased  their  reliance  on  market  research  data;  judgment-oriented 
creative  staff  decreased  their  reliance  on  the  same  data. 


induced  by  scenarios;  in  the  absence  of  scenarios,  accountability  will  lead  to 
conservatism. 


65 


Method 


Subjects  and  Design 

Subjects  were  undergraduate  students  who  participated  for  extra  credit  in  an 
introductory  marketing  class.  Each  subject  was  randomly  assigned  to  one  of  five 
cells  in  a 2 (Scenario)  x 2 (Accountability)  between-subjects  design  (with  counter- 
balancing). A total  of  190  subjects  participated,  with  the  number  in  each  cell  ranging 
from  25  to  33. 

Materials  and  Procedure 


All  experimental  materials  took  the  form  of  surveys  distributed  to  each 

subject.  Subjects  worked  through  the  materials  at  their  own  pace.  Subjects  played 

the  role  of  a new  product  manager  and  were  asked  to  provide  their  “gut  reaction”  to 

several  new  product  ideas.  Some  subjects  then  imagined  and  wrote  a scenario 

describing  the  future  success  of  the  target  new  product  (the  Electronic  Book).  The 

scenario  manipulation  was  incorporated  into  the  role-playing  task  as  follows: 

This  stage  of  the  new  product  process  asks  you  to  imagine  the  idea’s  potential 
success...  In  doing  so,  you  may  find  that  you  need  to  fill  in  some  details 
regarding  the  product,  how  it  might  be  used,  its  features,  and  its  appeal  to 
customers.  For  example,  what  would  it  take  for  this  new  product  to  be 
successful?  What  would  it  be  like  to  use  this  new  product?  What  would  the 
world  be  like  if  this  new  product  were  widely  available  and  popular?  Use 
your  imagination  to  form  a vision  of  the  future  success  of  the  EBook. 

In  the  scenario  conditions,  subjects  received  an  accountability  manipulation 

before  the  scenario  generation  task,  before  judging,  or  not  at  all;  in  the  no-scenario 
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condition,  subjects  received  an  accountability  manipulation  either  before  judging  or 
not  at  all.  The  wording  of  the  accountability  manipulation  was  also  consistent  with 
the  role-playing  task: 

You  may  also  be  called  upon  to  explain  or  justify  your  position  to  ‘senior 
management’  (in  a brief  interview  with  the  experimenter  at  the  end  of  this 
session). 

A question  at  the  end  of  the  task  asked  subjects  to  report  whether  they  recalled  this 
manipulation  and  also  advised  subjects  that  interviews  would  only  occur  if  time 
permitted  at  the  end  of  the  session  (to  maintain  the  fiction  of  accountability). 

All  subjects  then  performed  a “feasibility  assessment”  and  generated  pro  and  con 
reasons  regarding  the  success  of  this  new  product  idea.  As  in  prior  studies,  subjects 
then  made  a series  of  predictions  that  included  probability  of  success  over  time, 
market  share  over  time,  launch  recommendation,  and  confidence. 

Results  and  Discussion 

Manipulation  Checks 

At  the  beginning  of  the  survey,  a screening  question  was  included  that  asked 
subjects  to  indicate  their  initial  reaction  to  the  new  product  idea.  Subjects  who  did 
not  indicate  a positive  reaction  were  eliminated  from  subsequent  analyses1^ 

(reflecting  the  real-world  notion  that  managers  consider  only  new  product  ideas  that 


15  Thirty-eight  subjects  indicated  a negative  initial  reaction  and  were  eliminated  from  all  analyses.  In 
addition,  probability  estimates  must  logically  increase  overtime;  probability  estimates  from  22 
subjects  that  did  not  meet  this  criteria  were  omitted.  The  distribution  of  eliminations  across  conditions 
did  not  appear  to  follow  any  pattern  for  eliminations  based  on  initial  reactions.  For  probability  logic 
checks,  eliminations  were  less  evenly  distributed.  However,  the  key  analysis  is  also  conducted  on 
other  dependent  variables  with  these  subjects  included.  Because  the  results  are  consistent,  the 
eliminations  do  not  bias  the  conclusions.  Unless  otherwise  indicated,  any  other  differences  in  cell 
sizes  during  hypothesis  testing  are  due  to  missing  response  data. 
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seem  plausible  on  the  surface).  At  the  end  of  the  survey,  subjects  were  asked  if  they 
recalled  the  accountability  manipulation.  The  accountability  manipulation  had  the 
predicted  effect  on  responses  to  the  manipulation  check  (x  =28.40,  p=.0001). 
Dependent  Variables 

An  initial  analysis  treated  the  data  as  a 2 (Scenario)  x 2 (Accountability) 
design  and  combined  conditions  in  which  subjects  received  the  accountability 
manipulation  before  or  after  imagining  the  scenario.17  A MANOVA  of  predictions 
for  probability  of  success  and  market  share  over  time  as  a function  of  scenario 
generation  and  accountability  was  conducted.  This  analysis  revealed  a significant 
two-way  interaction  (F i.i23=3.55,  p=.062).  This  finding  is  central,  because  it 
indicates  that  the  overall  effect  of  accountability  on  judgment  depends  upon  whether 
subjects  generated  scenarios.  To  explore  this  result  further,  MANOVAs  for  each 
prediction  variable  were  also  conducted.  The  interactions  were  found  to  be 


16  Of  those  students  who  received  the  accountability  manipulation,  83%  correctly  recalled  it;  57%  of 
students  who  did  not  receive  the  accountability  manipulation  check  correctly  responded  that  they  did 
not.  (Incorrect  recall  may  arise  from  an  affirmative  response  bias.  The  response  errors,  if  reflecting  a 
genuine  feeling  of  heightened  accountability,  would  actually  weaken  the  chance  of  finding  a 
significant  effect  of  accountability  by  reducing  the  strength  of  the  accountability  manipulation.)  All 
subjects  were  retained  in  subsequent  analyses. 

17  A MANOVA  of  predictions  for  probability  of  success  and  market  share  over  time  was  conducted  as 
a function  of  when  subjects  received  the  accountability  manipulation  in  the  scenario  condition  (order 
was  counter-balanced).  In  all  cases,  the  order  effect  was  non-significant,  which  justifies  collapsing 
across  order. 

18  The  two-way  accountability  X scenario  interaction  was  qualified  by  an  interaction  with  time  and  the 
prediction  variable  (cubic-time  F,  ,23=3.70,  p=.057).  This  higher-order  interaction  reflects  the  notion 
that  the  strength  of  the  two-way  interaction  of  scenario  and  accountability  depends  upon  the  prediction 
variable  and  the  time  period.  Practically  speaking,  the  probability  and  market  share  predictions  were 
cumulative  and  non-cumulative  (respectively),  so  it  would  not  be  surprising  if  a ‘levelling  off 
occurred  in  more  distant  time  periods  for  market  share  but  not  probability.  In  fact,  there  is  a cubic- 
time interaction  for  market  share  (F|  144=3. 67,  p=.057)  but  not  for  probability  of  success  (F,  i24=2.00, 
p=.  160). 
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significant  for  both  probability  of  success  over  time  (Fiji24=3.31,  p=.071)  and  market 
share  (Fi. 144=3. 46,  p=.065).  Follow-up  tests  of  the  simple  effects  of  accountability 
were  also  examined  for  each  prediction  variable.  For  probability  of  success 
estimates,  accountability  led  to  more  optimistic  judgments  when  scenarios  were 
generated  (Fpi24=4.58,  p=.034);  otherwise,  accountability  had  no  effect  (F<1).  For 
market  share  estimates,  accountability  led  to  marginally  more  conservative  estimates 
when  scenarios  were  not  generated  (Fiii44=2.63,  p=.107);  otherwise,  accountability 
had  no  effect  (F<1).  The  overall  pattern  of  means  is  consistent:  without  scenarios, 
accountability  led  to  more  conservative  judgments;  with  scenario  generation, 
accountability  exacerbated  optimism.  This  pattern  is  evident  in  the  slopes  of  each  of 
the  lines,  or  more  simply  the  average  prediction  mean  across  all  periods  (see  Tables 
1 1 and  12  and  Figure  6). 

In  addition  to  these  results  on  the  key  dependent  measures,  analyses  of  several 
related  dependent  variables  provide  additional  support.  For  example,  subjects  were 
asked  to  predict  how  long  it  would  take  “for  the  first  Electronic  Book  to  be  launched 
nationally  in  the  USA”  and  “for  over  1 million  people  to  own  an  Electronic  Book.” 
There  is  a significant  two-way  interaction  of  accountability  and  scenarios 
(Fi, 146=3. 92,  p=. 050),  qualified  by  an  interaction  with  the  prediction  variable 
(Fi, 146=4.909,  p=.028). 19  As  the  means  in  Table  13  indicate,  subjects  who  did  not 
generate  scenarios  predicted  a longer  time  period  if  held  accountable  (a  conservatism 


19  Follow-up  univariate  analyses  reveal  non-significant  effects  for  time-to-launch  (F<1)  but  significant 
effects  for  time-to-1 -million-ownership  (F,  i46=4.99,  p=.027).  In  simple  effects  tests,  accountability 
led  to  more  conservative  predictions  when  scenarios  were  not  generated  (F|  |46=3.  69,  p=.057);  the 
effect  of  accountability  was  non-significant  when  scenarios  were  generated  (F|  j46=1.39,  p=.241). 
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bias);  in  contrast,  subjects  who  did  generate  scenarios  predicted  a shorter  time  period 
under  accountability  (an  optimism  bias).  The  effect  is  particularly  pronounced  for 
predictions  regarding  the  1 million  ownership  mark,  compared  to  launch. 

Overall,  these  results  suggest  that  the  accountability  of  real-world  managers 
may  not  improve  nonanalytic  judgment.  It  can  be  viewed  as  a relatively  conservative 
test,  inasmuch  as  the  scenario  task  was  only  a few  minutes  long  and  all  subjects  also 
generated  evenhanded  pro  and  con  reasons  prior  to  judgment.  Consistent  with  all  of 
the  studies  thus  far,  generating  evenhanded  reasons  seems  insufficient  to  improve 
judgment  after  nonanalytic  thinking.  Worse  still,  accountability  may  exacerbate  a 
scenario-induced  bias  by  increasing  its  resistance  to  evenhanded  reasoning.  Perhaps 
accountability  motivated  subjects  to  think  harder,  not  smarter,  and  try  to  ‘tell  a better 
story’.  In  doing  so,  generating  a more  elaborate  scenario  may  have  exacerbated 
scenario-induced  bias,  consistent  with  the  findings  of  Pilot  Study  2.  Interestingly, 
Sieck,  Quinn,  and  Schooler  (1999)  recently  demonstrated  that  justification  leads  to 
polarization  of  analogy  judgments.  These  findings  are  particularly  disconcerting 
inasmuch  as  accountability  is  generally  thought  to  improve  judgment  by  inducing 
evenhanded  and  analytic  thought.  In  contrast,  accountability  appears  to  exacerbate 
one-sidedness  in  nonanalytic  thinking. 
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Table  11. 


Study  2:  probability  of  success  estimates  over  time  as  a function  of  scenarios  and 
accountability  for  electronic  books. 


Scenario 

Account- 

ability 

N 

Next  5 
years 

Next  10 
years 

Next  20 
years 

Beyond 
20  years 

Average 

No 

Not 

24 

19.71 

44.13 

69.17 

88.96 

54.86 

Scenario 

Accountable 

(11.79) 

(17.78) 

(15.16) 

(12.14) 

(10.79) 

No 

Accountable 

27 

20.76 

40.87 

65.22 

85.93 

53.19 

Scenario 

(18.86) 

(20.92) 

(20.68) 

(15.13) 

(17.04) 

Scenario 

Not 

26 

13.38 

35.04 

64.38 

83.81 

49.15 

Accountable 

(10.04) 

(15.85) 

(20.37) 

(16.02) 

(13.32) 

Scenario 

Accountable 

52 

21.06 

44.52 

68.99 

91.66 

56.56 

(19.63) 

(21.97) 

(20.00) 

(10.60) 

(15.16) 

Table  12. 


Study  2:  market  share  estimates  over  time  as  a function  of  scenarios  and 
accountability  for  electronic  books. 


Scenario 

Account- 

ability 

N 

2000- 

2005 

2005- 

2010 

2010- 

2020 

Beyond 

2020 

Average 

No 

Not 

25 

23.00 

43.60 

70.20 

80.80 

53.70 

Scenario 

Accountable 

(14.07) 

(16.99) 

(20.13) 

(23.83) 

(11.86) 

No 

Accountable 

30 

28.07 

38.33 

53.50 

69.50 

47.98 

Scenario 

(29.93) 

(21.73) 

(21.98) 

(29.43) 

(17.26) 

Scenario 

Not 

32 

24.09 

41.87 

57.19 

73.36 

49.94 

Accountable 

(19.29) 

(22.17) 

(22.75) 

(28.37) 

(15.44) 

Scenario 

Accountable 

62 

27.69 

43.90 

60.77 

77.63 

52.50 

(24.42) 

(20.51) 

(22.55) 

(29.21) 

(16.52) 

71 
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Figure  6. 

Study  2:  average  probability  of  success  estimates  (A)  and  market  share  (B)  as  a 
function  of  scenarios  and  accountability  for  electronic  books. 


Table  13. 


Study  2:  predictions  for  launch  and  target  sales  as  a function  of  scenarios  and 
accountability  for  electronic  books. 


Scenario 

Account- 

ability 

N 

# of  years  until 
national  launch 

# of  years  until  1- 
million  own  Ebooks 

No 

Scenario 

Not 

Accountable 

25 

4.02  (2.96) 

9.20  (5.43) 

No 

Scenario 

Accountable 

30 

4.83  (4.13) 

13.13  (10.30) 

Scenario 

Not 

Accountable 

32 

4.41  (4.62) 

11.76  (7.75) 

Scenario 

Accountable 

62 

3.99  (3.48) 

9.75  (6.84) 

CHAPTER  7 
ANALOGIES 

Analogies  are  another  potential  source  of  bias  in  judgment  and  will  be  the 
focus  of  the  second  part  of  this  dissertation.  An  analogy,  of  course,  is  a comparison 
of  two  objects  on  certain  dimensions,  usually  for  the  purpose  of  inference  making. 
Reasoning  by  analogy  about  a target  appears  to  consist  of  several  steps:  access  or 
retrieval  of  a base  analog,  mapping  of  the  base  and  target,  inference  making,  and 
perhaps  higher  order  learning  (e.g.,  Holyoak  & Thagard  1997;  Gregan-Paxton  & 
Roedder  John  1 997).  Prior  research  on  analogies  has  focused  on  factors  that 
influence  each  step  of  reasoning  by  analogy.  Evidence  suggests  that  analogical 
retrieval  and  transfer  is  influenced  by  structural  relations,  surface  details,  problem 
context  and  goals,  although  their  relative  importance  is  a matter  of  continued  debate 
(for  reviews,  see  Reeves  & Weisberg  1994;  Gregan-Paxton  & Roedder  John  1997). 
The  influence  of  structural  relations  reflects  causal  reasoning  and  analytic  thinking; 
the  influence  of  surface  details,  similarity-based  inferences,  and  other  salient  non- 
causal  information  reflects  nonanalytic  thinking. 

Prior  research  suggests  that  the  retrieval  stage  may  be  especially  susceptible  to 
surface  and  contextual  details  that  serve  as  cues  to  retrieval.  For  example,  subjects 
are  more  likely  to  spontaneously  retrieve  a close  analogy  from  a medical  domain  than 
a remote  analogy  from  a military  domain  in  generating  a solution  to  a medical 
problem  (Gick  & Holyoak  1980).  As  with  priming  effects,  people  may  be  unaware 
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of  the  analogy’s  effect  on  judgment  (Schunn  & Dunbar  1996).  The  effects  of 
superficial  similarities  in  reasoning  by  analogy  can  have  dramatic  effects  on 
judgment.  For  example,  people  are  more  likely  to  recommend  direct  military 
intervention  in  a conflict  scenario  when  irrelevant  details  (e.g.,  refugees  fleeing  in 
boxcars  versus  small  boats)  cue  an  analogy  to  World  War  II  versus  Vietnam 
(Gilovich  1981).  Similarly,  people  who  judge  the  likely  success  of  a rookie  football 
player  are  biased  by  irrelevant  information  (e.g.,  hometown)  in  an  analogy  to  a 
successful  professional  player.  Gilovich’s  result  is  particularly  notable  because  the 
irrelevant  analogy  appears  to  bias  judgment  for  both  experts  and  novices,  although 
some  researchers  speculate  that  these  effects  may  be  amplified  for  novices  (e.g., 
Gregan-Paxton  & Roedder  John  1997).  In  these  examples,  the  effect  of  surface 
details  originate  during  the  retrieval  stage  and  carried  through  mapping  and  inference 
making.  Holyoak  and  Thagard  (1995a)  note:  “Too  often  people. ..make  use  of 
superficially  appealing  analogies  to  highly  familiar  source  analogs,  which  actually 
miss  most  of  the  important  causal  relations  in  the  target...  People  sometimes  latch  on 
to  a salient  analogy  and  focus  on  it  to  the  neglect  of  deeper  thought  about  the  relevant 
problem,  decision,  or  explanation.” 

Prior  research  suggests  that  people  do  try  to  be  rational  when  reasoning  by 
analogy.  For  example,  people  prefer  to  generate  analogical  mappings  that  honor 
basic  coherence  constraints  (Spellman  & Holyoak  1992).  People  also  prefer  to  make 
inferences  based  on  systematic  correspondence  between  the  target  and  the  analogy 
(Markman  1997).  Thus,  knowledge  in  the  analogy’s  domain  influences  inference 
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making  in  the  target  domain.  For  example,  in  a study  by  Spellman  and  Holyoak 
(1992),  people’s  mappings  between  World  War  II  and  the  Persian  Gulf  War  were 
influenced  by  their  knowledge  of  World  War  II;  inferences  about  the  Persian  Gulf 
War  were  altered  by  emphasizing  different  aspects  of  World  War  II.  Mappings  are 
also  influenced  by  goals.  For  example,  subjects  tend  to  resolve  ambiguous  mappings 
in  a manner  that  favors  their  goals  (Spellman  & Holyoak  1996). 

Thus,  reasoning  by  analogy  is  a fallible  process.  Inferences  are  typically 
generated  by  aligning  the  base  and  target  and  copying  relations  from  the  base  to  the 
target.  The  relations  are  known  to  be  true  about  the  base  but  are  not  necessarily  true 
about  the  target.  However,  the  analogy  lends  support  or  plausibility  to  these  target 
inferences.  For  example,  people  judge  a target  explanation  of  an  event  as  more  likely 
when  a consistent  analogy  is  provided  (Read  & Cesa  1991).  In  addition,  the  analogy 
also  emphasizes  some  dimensions  of  the  problem  at  the  expense  of  others  by  focusing 
attention  on  the  mapped  rather  than  unmapped  aspects  of  the  target  (Gentner  & 
Markman  1 997).  As  a result,  even  appropriate  analogies  (based  on  relevant 
dimensions)  can  be  over-applied,  leading  to  neglect  of  other  dimensions  or 
inappropriate  inferences.  In  other  words,  the  analogy  may  serve  as  a mental  model  or 
focal  hypothesis  that  drives  elaboration  in  a relatively  one-sided  manner.  Zaltman 
(1997)  argues  more  generally  that  metaphor  guides  attention  and  perception,  elicits 
hidden  knowledge  and  serves  as  the  “engine  of  imagination.”  Analogies  may  be 
particular  enticing  in  uncertain  or  complex  judgments  because  they  provide  a 
coherent  interpretation  of  a focal  problem,  i.e.,  local  consistency  at  the  expense  of 
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global  consistency  (Klayman  & Schoemaker  1993).  If  this  is  the  case,  then 
elaboration  may  lead  to  one-sided  reasoning  by  analogy,  thereby  reinforcing  the 
analogy  and  polarizing  judgment.  Evidence  for  reinforcement  was  found  in  a study 
by  Sieck,  Quinn,  and  Schooler  (1999).  Justification  (a  special  form  of  elaboration) 
polarized  analogy  judgments,  and  subjects  appeared  to  engage  in  a one-sided  search 
for  commonalities  rather  than  differences  in  justifying  their  judgments. 

In  the  absence  of  hard  data  and  causal  evidence,  new  products  judgments 
may  be  especially  susceptible  to  reasoning  by  analogy.  The  question  then  turns  to 
whether  the  analogy  is  appropriate.  Presumably  analogies  based  on  superficial 
similarity,  either  surface  or  global,  are  not  necessarily  useful  guides  to  judgment. 

Even  if  the  analogy  is  relevant  on  some  dimensions,  elaboration  may  lead  to 
inappropriate  inference  making  on  other  dimensions.  Based  on  Gilovich’s  strong 
evidence,  as  well  as  arguments  drawn  from  the  literature  on  reasoning  by  analogy,  I 
propose  that  analogies  (like  scenarios)  can  bias  judgment.  This  notion  underlies 
Proposition  1 , and  my  research  is  designed  to  provide  evidence  of  analogy  effects  in  a 
new  product  context.  Furthermore,  analogies  may  be  important  when  the 
effectiveness  of  analytic  tools  are  examined.  I am  unaware  of  any  direct  findings  in 
prior  work  on  analogies  that  test  whether  their  effects  can  be  eliminated  by  analytic 
thinking.  The  resistance  of  analogies  to  subsequent  analytic  thought  is  the  basis  of 
Proposition  2 and  will  be  investigated  in  a series  of  studies. 


CHAPTER  8 

PILOT  STUDY  3:  ANALOGIES  AND  ELABORATION 
The  first  objective  of  this  study  is  to  provide  an  initial  demonstration  of  the 
effects  of  analogies  and  their  resistance  to  subsequent  analytic  thinking.  The  study  is 
also  designed  to  investigate  the  effects  of  elaboration  on  analogy-induced  biases. 

As  previously  discussed,  some  research  suggests  that  analogy-induced  biases 
may  be  the  result  of  relatively  superficial  reasoning  by  analogy  (Gilovich  1981).  On 
the  other  hand,  some  researchers  suggest  that  people  prefer  analogical  mappings  that 
are  rational  and  based  more  on  deep  structure  than  surface  similarities,  although  their 
relative  importance  is  a matter  of  continued  debate  (Reeves  & Weisberg  1994).  The 
(unstated)  implication  seems  to  be  that  if  people  were  to  elaborate  more  deeply  on  an 
analogy,  they  might  recognize  that  its  similarity  is  superficial  and  then  correct  for  the 
bias.  At  the  very  least,  additional  thought  should  lead  to  consideration  of  the  deeper 
structure  of  the  problem  and  induce  more  analytic  thinking.  If  this  is  the  case,  then 
elaboration  may  reduce  the  effects  of  a superficial  analogy. 

On  the  other  hand,  Sieck,  Quinn,  and  Schooler  (1999)  provide  some  evidence 
that  subjects  weight  surface  similarity  more  heavily  when  asked  to  justify  their 
judgments  (instead  of  shifting  to  heavier  weighting  of  the  analogy’s  deep  structure). 
Moreover,  judgments  of  the  analogy  polarized  as  a result  of  justification,  and  subjects 
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appeared  to  consider  commonalities  rather  than  differences  in  a one-sided  manner.20 
I have  previously  argued  that  analogies  lead  to  relatively  one-sided  elaboration  and 
the  development  of  a schema  that  reinforces  the  initial  analogy.  Insofar  as  reasoning 
by  analogy  tends  to  reinforce  the  analogy,  it  could  account  for  the  persistence  of 
analogy-induced  biases  over  time.  This  hypothesis  is  consistent  with  prior  research 
on  polarization  (e.g.,  Millar  & Tesser  1986),  which  suggests  that  additional  thought 
may  serve  to  bolster  a schema  and  polarize  judgments.  Similarly,  adopting  and 
elaborating  an  analogy  may  also  induce  directional  thought  and  lead  to  polarized 
judgment. 

This  study  is  designed  to  investigate  the  effects  of  elaboration  on  analogy- 
induced  biases.  Recall  that  Pilot  Study  2 and  Study  2 investigated  the  effects  of 
elaboration  (directed  and  under  accountability)  on  scenario-induced  biases.  This 
study  provided  subjects  with  an  analogy  followed  by  a relatively  neutral  elaboration 
instruction.  Doing  so  provided  a relatively  stringent  test  of  the  effects  of  elaboration 
on  judgment  because  subjects  were  free  to  elaborate  in  an  evenhanded  manner  if  they 
so  choose.  In  addition,  subjects  were  instructed  to  generate  evenhanded  reasons  in  an 
attempt  to  induce  analytic  thinking  prior  to  judgment.  Thus,  ample  opportunity  for 
elaboration  to  reduce  the  superficial  effects  of  an  analogy  was  included  in  the  study’s 
design  in  order  to  address  its  hypotheses.  That  is,  an  initial  analogy  biases  judgment 
and  subsequent  elaboration  (via  a neutral  scenario  and  evenhanded  reasons)  does  not 
eliminate  its  effects. 


20  My  interest  lies  in  the  analogy’s  effect  on  forecasts,  rather  than  judgments  of  the  analogy  itself. 


78 


Method 

Subjects  and  Design 

Subjects  were  undergraduate  students  who  participated  for  extra  credit  in  an 
introductory  marketing  class.  Each  subject  was  randomly  assigned  to  one  ol  four 
cells  in  a 2 (Scenario  versus  not)  x 2 (Analogy:  positive/negative)  x 2 (Repeat 
predictions:  before  and  after  pro/con  generation)  mixed  design.  A total  of  1 09 
subjects  participated,  with  the  number  in  each  cell  ranging  from  20  to  3 1 . 

Materials  and  Procedure 

All  experimental  materials  took  the  form  of  surveys  distributed  to  each 

subject.  Subjects  worked  through  the  materials  at  their  own  pace.  The  survey  asked 

subjects  to  play  the  role  of  a business  executive  and  think  about  the  future  of 

Electronic  Books.  Subjects  were  given  some  background  information  on  the  EBook 

and  were  asked  to  give  an  initial  professional  opinion.  Subjects  were  then  given  an 

analogy  to  the  success/fast  growth  of  cellular  phones  or  the  failure/slow  growth  of 

videophones.  In  the  success  analogy  condition,  subjects  were  instructed  as  follows: 

Please  think  about  the  future  of  Electronic  Books.  In  doing  so,  you  may 
find  it  helpful  to  consider  the  case  of  cellular  telephones  since  some 
people  believe  that  the  Ebook  could  be  as  successful  in  the  marketplace  as 
cellular  phones. 

In  the  failure  analogy  condition,  subjects  received  the  following  instruction: 

Please  think  about  the  future  of  Electronic  Books.  In  doing  so,  you  may 
find  it  helpful  to  consider  the  case  of  video  telephones  since  some  people 
believe  that  the  EBook  could  be  as  unsuccessful  in  the  marketplace  as 
videophones. 


79 


Subjects  were  also  provided  with  numerical  information  and  a graph  of  cumulative 

sales  over  time  reflecting  25%  penetration  over  10  years  for  cellular  phones  in  the 

USA  (in  the  success  analogy  conditions)  or  less  than  1%  penetration  over  10  years  for 

videophones  in  the  USA  (in  the  failure  analogy  conditions). 

Some  subjects  then  generated  a scenario  describing  the  future  of  Ebooks.  The 

scenario  instructions  were  intended  to  be  neutral: 

Please  take  some  time  to  think  about  a scenario  describing  the  future  of 
Electronic  Books.  (Recall  the  characteristics  of  a scenario  - it  is  a story 
that  describes  how  events  unfold  over  time  and  includes  vivid  details  that 
create  a mental  picture  or  vision  of  the  future.)  Write  a scenario  that 
describes  the  future  of  the  EBook. 

Other  subjects  proceeded  to  the  next  section  of  the  booklet. 

Subjects  then  responded  to  a series  of  questions  concerning  the  future  of 

EBooks,  including  their  opinion,  probability  of  success  of  EBooks,  adoption  curve 

predictions  over  time,  and  likelihood  and  confidence  measures  (time  1).  Subjects 

then  generated  pro  and  con  reasons  for  the  future  of  EBooks  and  responded  to  several 

measures  again  (time  2). 

Results  and  Discussion 

This  study  investigates  whether  an  analogy  biases  judgment  and  whether 
elaboration  (either  via  scenario  generation  or  pro/con  generation)  can  effectively 
debias  judgment.  A main  effect  of  the  analogy  on  judgment  would  indicate  that  the 
analogy  induced  bias.  Depending  on  when  the  dependent  measure  is  taken,  it  is  also 
possible  to  test  for  the  effectiveness  of  elaboration  in  correcting  for  this  bias. 
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First,  the  results  consistently  demonstrate  an  analogy-induced  bias.  There  is  a 
significant  main  effect  of  the  analogy  on  judgments  made  at  time  1 (see  Table  14). 
Subjects  who  were  given  an  initial  negative  analogy  changed  their  professional 
opinion  in  a more  negative  direction  than  subjects  who  were  given  a positive  analogy 
(Fi,io3=6.92,  p=.0098).  In  addition,  the  positive  analogy  led  to  more  favorable 
judgments  of  probability  of  success  (Fi  io4=6.42,  p=.0 1 3),  increased  choice  of  success 
over  failure  as  best  describing  the  future  of  EBooks  (x2=4.73,  p=.030),  and  greater 
confidence  in  the  success  of  EBooks  (Fj . io4=4.5  1 , p=.036).  The  difference  in 
likelihood  ratings  given  to  best-case  and  worst  case  future  scenarios  was  smaller 
when  subjects  were  given  negative  versus  positive  analogies  (Fpioo=3.07,  p=.083). 
Furthermore,  subjects  who  were  given  a positive  analogy  were  more  likely  to  choose 
an  adoption  curve  with  the  most  rapid  growth  as  best  fitting  the  future  of  the  EBook 
(X2=3.79,  p=,052). 

Whether  subjects  elaborated  via  scenarios  had  no  effect  on  any  of  the 
judgments  described  above  (both  main  effect  of  scenarios  and  interaction  with 

21 

analogies  are  non-significant  for  all  prediction  variables).  These  consistent  findings' 
suggest  that  an  initial  analogy  biases  judgment  and  that  elaborating  on  a future 
scenario  does  not  undo  the  biasing  effects  of  the  initial  analogy.  There  is  no  evidence 
that  scenario  elaboration  exacerbates  the  initial  bias. 


21  Only  one  result  deviates  from  this  finding.  There  are  no  significant  effects  on  predictions  for  the 
adoption  curve  (percentage  of  people  who  have  adopted  the  Ebook  in  2003,  2008,  2013  and  beyond) 
made  by  subjects  at  time  1,  and  the  direction  of  the  means  is  not  entirely  consistent.  (The  slope  is 
directionally  steeper  for  the  positive  versus  negative  analogy  in  the  no-scenario  condition,  but  not  in 
the  scenario  condition.)  This  finding  is  discussed  in  more  detail  when  I examine  the  effects  of 
generating  pro  and  con  reasons  on  repeated  judgments. 
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Given  the  repeated  measures  design,  it  is  also  possible  to  investigate  the 
effects  of  generating  pro  and  con  reasons  on  judgments  that  have  been  biased  by  an 
initial  analogy.  After  elaborating  via  a scenario  and  then  via  generating  pro  and  con 
reasons,  the  positive  analogy  still  led  to  more  favorable  final  judgments  of  probability 
of  success  than  the  negative  analogy  (see  Table  15).  Indeed,  a MANOVA  of  both 
time  1 and  time  2 probability  judgments  reveals  a significant  main  effect  of  analogy 
(Fi, io4=4. 63,  p=.034)  and  a non-significant  interaction  with  repeated  judgment,  which 
suggests  that  the  pro-con  generation  did  not  improve  judgment  on  this  prediction 
variable  (all  changes  are  non-significant).  Similarly,  subjects  who  were  given  an 
initial  negative  analogy  still  changed  their  opinion  in  a more  negative  direction  than 
subjects  who  were  given  a positive  analogy  (Fiiio3=3.42,  p=.067).  Again,  a 
MANOVA  of  both  attitude  change  variables  (constructed  from  initial,  time  1 and 
time  2 judgments;  see  Table  16  and  Figure  7)  reveals  a significant  main  effect  of 
analogy  (Fi, 103=6. 43,  p=.013)  and  a non-significant  interaction  with  repeated 
judgment.  Thus,  there  is  no  evidence  that  generating  pro  and  con  reasons  had  any 
debiasing  effect  on  judgment. 

Interestingly,  there  appears  to  be  some  evidence  that  the  pro-con  generation 
task  may  have  exacerbated  the  analogy-induced  bias.  Predictions  at  time  2 for  the 
adoption  curve  reveal  a significant  interaction  of  the  analogy  with  time  (quadratic 
Fi,  104=7. 04,  p=009).  This  interaction  reflects  predictions  of  slower  growth  by 
subjects  given  the  negative  versus  positive  analogy,  as  illustrated  in  Figure  8. 
However,  this  same  analysis  for  predictions  at  time  1 revealed  no  significant  effects. 
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A MANOVA  of  predictions  at  time  1 and  time  2 reveals  an  overall  significant 
interaction  of  analogy  and  repetition  and  estimation  period  (quadratic  polynomial 
Fi.  104=6. 92,  p=. 0098).  This  interaction  indicates  that  subjects  change  their  predictions 
after  generating  pro  and  con  reasons  in  the  direction  favored  by  the  initial  analogy, 
i.e.,  toward  slower  growth  if  given  a negative  analogy. 

Another  way  of  examining  the  effect  of  generating  pro  and  con  reasons  is  to 
analyze  the  change  in  adoption  predictions  from  time  1 to  time  2 (see  Table  1 7).  A 
MANOVA  of  these  change  measures,  constructed  by  calculating  the  difference 
between  time  1 and  time  2 judgments,  reveals  that  the  effect  of  the  analogy  depends 
on  the  estimation  period  (F35io2=3  .263 , p=.025).  Follow-up  univariate  analyses  of  the 
changes  for  each  estimation  period  indicate  that  the  effects  of  the  analogy  are 
primarily  due  to  its  impact  on  the  change  in  prediction  for  the  year  2008  (Fi 304=5.73, 
p=.019).  This  is  consistent  with  the  quadratic  interaction  above:  after  generating  pro 
and  con  reasons,  subjects  given  a negative  analogy  lowered  their  estimates  for  the 
year  2008,  thereby  predicting  a ‘slower  growth’  adoption  curve,  compared  to  subjects 
given  a positive  analogy.  Thus,  the  attempt  to  debias  judgment  by  asking  subjects  to 
generate  pro  and  con  reasons  did  not  improve  judgment  and  perhaps  may  have 
exacerbated  the  analogy-induced  bias  in  some  of  these  forecasts.  This  result  provides 
partial  support  for  the  exacerbating  effects  of  elaboration,  occuring  not  during  initial 
elaboration  via  the  scenario  but  instead  between  time  1 and  time  2 (across  the 
additional  questions  and  evenhanded  reason  generation  task). 
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Overall,  the  results  of  this  study  provide  support  for  the  hypotheses. 

Judgments  were  biased  by  an  initial  analogy.  This  result  is  consistent  with  prior 
research  on  the  robust  phenomenon  of  anchoring  (e.g.,  Wilson,  Houston,  Etling,  & 
Brekke  1996),  a distinction  that  I will  address  in  the  next  study.  Moreover,  the  effects 
of  an  analogy  endured  despite  subsequent  elaboration.  Elaboration  via  a scenario  did 
not  reduce  the  biasing  effects  of  the  initial  analogy,  which  suggests  that  deeper 
processing  of  an  analogy  is  not  necessarily  more  analytic.  On  the  other  hand,  there 
was  little  support  for  the  notion  that  elaboration  exacerbates  bias  (because  the 
scenario  effect  was  non-significant).  In  addition,  elaboration  via  evenhanded 
reasoning  did  not  eliminate  the  analogy-induced  bias,  and  there  was  even  some 
evidence  suggesting  that  judgment  polarized  after  pro/con  reason  generation.  Due  to 
the  experimental  design,  however,  it  is  unclear  whether  polarization  occurred  during 
the  evenhanded  reasoning  task  or  during  questioning.  For  example,  prior  research  has 
demonstrated  that  witness’  confidence  in  their  testimony  is  susceptible  to  influence 
by  post-event  questioning  (Shaw  1996).  In  this  study,  changes  in  judgment  from  time 
1 to  time  2 could  have  arisen  from  intervening  questions;  insofar  as  those  questions 
prompt  elaboration  and  inference-making  driven  by  the  initial  analogy,  then  bias  in 
judgment  may  be  exacerbated.  Certainly,  there  was  no  evidence  that  the  effect  of  the 
initial  analogy  was  eliminated,  or  even  reduced,  by  evenhanded  reasoning  (which 
supports  the  main  hypothesis  of  this  study). 
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Table  14. 


Pilot  study  3:  predictions  for  future  of  electronic  books  (at  time  1)  as  a function  of 
analogy  and  scenario  elaboration. 


Scenario 

Analogy 

Subjects 
choosing 
graph  with 
fastest 
growth 

Subjects 

choosing 

“Best- 

Case” 

future 

Confidence 
in  Choice 
of  Best- 
Case  future 

Difference  in 
Scenario 

Likelihood  Ratings 
(Best-Worst  Case) 

No 

Negative 

5/23 

16/23 

55.83 

1.26 

Scenario 

(22%) 

(70%) 

(30.55) 

(2.83) 

No 

Positive 

9/20 

17/20 

67.00 

2.05 

Scenario 

(45%) 

(85%) 

(25.46) 

(2.74) 

Scenario 

Negative 

10/33 

25/33 

64.85 

1.58 

(40%) 

(76%) 

(20.48) 

(3.04) 

Scenario 

Positive 

14/32 

30/32 

72.34 

2.68 

(44%) 

(94%) 

(13.44) 

(2.12) 

Table  15. 


Pilot  study  3:  probability  of  success  of  electronic  books  as  a function  of  analogy  and 
scenario  elaboration  before  and  after  reason  generation. 


Scenario 

Analogy 

N 

Probability  at  time  1 
(after  analogy  and 
scenario  manipulation) 

Probability  at  time  2 
(after  generating  pros 
and  cons) 

No  Scenario 

Negative 

23 

54.35  (21.71) 

53.91  (25.72) 

No  Scenario 

Positive 

20 

66.50  (14.24) 

66.00  (20.10) 

Scenario 

Negative 

33 

60.91  (17.74) 

63.64  (18.00) 

Scenario 

Positive 

32 

65.94  (14.56) 

64.38  (17.22) 
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Table  16. 

Pilot  study  3:  attitude  toward  electronic  books  as  a function  of  analogy  and  scenario 
elaboration  before  and  after  reason  generation. 


Scenario 

Analogy 

N 

Initial 

Attitude 

Attitude 
At  time 

1 

Attitude 
at  time  2 

Diff  1 

Diff  2 

No 

Scenario 

Negative 

23 

5.04 

(1.46) 

4.69 

(1.52) 

4.52 

(1.56) 

-0.34 

(0.57) 

p=.067 

-0.52 

(1.31) 

p=.016 

No 

Scenario 

Positive 

20 

4.95 

(1.14) 

5.20 

(1.15) 

4.85 

(1.35) 

+0.25 

(0.79) 

n.s. 

-0.10 

(0.85) 

n.s. 

Scenario 

Negative 

33 

5.30 

(1.02) 

5.03 

(1.33) 

4.88 

(1.19) 

-0.27 

(1.13) 

p=.085 

-0.42 

(1.03) 

p=.019 

Scenario 

Positive 

31 

5.03 

(1.02) 

5.10 

(1.04) 

4.94 

(0.96) 

+0.06 

(0.89) 

n.s. 

-0.10 

(0.87) 

n.s. 

Note:  Diff  1 (Diff  2)  is  the  difference  between  the  attitude  at  time  1 (time  2)  and  the 
initial  attitude.  P-values  are  reported  for  testing  whether  differences  are  non-zero. 
However,  the  possibility  of  drift  implies  that  contrasts  to  zero  are  an  inappropriate 
analysis. 


A 


B 


Figure  7. 

Pilot  study  3 : attitude  change  toward  electronic  books  after  initial  analogy  and 
scenario  elaboration  (A)  and  after  reason  generation  (B) 
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Table  17. 


Pilot  study  3:  adoption  curve  predictions  for  electronic  books  as  a function  of 
analogy  and  scenario  elaboration  before  and  after  reason  generation. 


Scenario 

Analogy 

N 

Adoption  Curve  Estimates  Before  Generating 
Reasons  (Time  1 ) 

2003  2008  2013  Beyond  2018 

No  Scenario 

Negative 

23 

16.52 

29.65 

41.61 

54.61 

(18.21) 

(23.11) 

(27.59) 

(30.91) 

No  Scenario 

Positive 

20 

14.56 

29.65 

45.10 

59.65 

(11.01) 

(22.00) 

(29.02) 

(34.15) 

Scenario 

Negative 

33 

15.88 

33.03 

49.36 

65.58 

(15.88) 

(19.90) 

(25.61) 

(28.52) 

Scenario 

Positive 

32 

14.59 

29.19 

45.16 

58.47 

(14.59) 

(21.29) 

(26.93) 

(29.46) 

Adoption  Curve  Estimates  After  Generating 

Reasons  (Time  2) 

No  Scenario 

Negative 

23 

12.70 

25.43 

38.52 

52.70 

(11.00) 

(19.76) 

(26.59) 

(31.45) 

No  Scenario 

Positive 

20 

14.50 

29.50 

44.65 

54.55 

(9.56) 

(20.03) 

(29.42) 

(34.14) 

Scenario 

Negative 

33 

13.47 

24.29 

39.45 

58.33 

(9.34) 

(14.20) 

(20.89) 

(28.11) 

Scenario 

Positive 

32 

13.22 

28.09 

39.88 

52.72 

(11.36) 

(19.80) 

(25.31) 

(28.56) 

Change  in  Adoptio 

m Curve  Estimates  with 

Repeated  Judgment 

No  Scenario 

Negative 

23 

-3.83 

-4.22 

-3.09 

-1.91 

(9.97) 

(9.08) 

(6.95) 

(8.51) 

p=.027 

P=-H 

n.s. 

n.s. 

No  Scenario 

Positive 

20 

-0.05 

-0.15 

-0.45 

-5.10 

(5.91) 

(13.44) 

(18.41) 

(12.49) 

n.s. 

n.s. 

n.s. 

p=.079 

Scenario 

Negative 

33 

-2.41 

-8.74 

-9.91 

-7.24 

(6.74) 

(14.66) 

(17.19) 

(14.97) 

"a 

II 

o 

-fc. 

p=.0001 

p=.0001 

p=.002 

Scenario 

Positive 

32 

-1.38 

-1.09 

-5.28 

-5.75 

(9.30) 

(11.28) 

(11.24) 

(13.21) 

n.s. 

n.s. 

r-921 

_.Ez.— l.j — 

Note:  P-values  indicate  whether  change  is  significantly  different  from  0.  However, 
the  possibility  of  drift  implies  that  contrasts  to  zero  are  an  inappropriate  analysis. 
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A B 

Figure  8. 

Pilot  study  3:  market  share  estimates  after  generating  reasons  for  electronic  books 
with  (B)  or  without  (A)  scenario  elaboration. 


CHAPTER  9 

PILOT  STUDY  4:  ANALOGY  EFFECTS  IN  A MORE  CONSERVATIVE  TEST 

The  previous  pilot  demonstrated  persistent  effects  of  analogies.  Subjects 
given  an  analogy  to  a successful  product  were  more  optimistic  about  a new  product 
than  subjects  given  an  analogy  to  an  unsuccessful  product.  This  difference  endured 
despite  subsequent  elaboration. 

Pilot  Study  4 serves  as  a more  conservative  test  of  analogy  effects  on 
judgment.  In  this  study,  some  subjects  were  provided  with  an  analogy;  others  were 
not.  All  subjects  received  the  same  information  about  the  target  new  product. 

Subjects  then  elaborated  by  generating  a scenario  or  reasons.  In  addition,  subjects 
were  presented  with  evenhanded  evidence  to  consider.  Thus,  ample  opportunity  for 
elaboration  to  reduce  the  superficial  effects  of  an  analogy  was  included  in  the  study’s 
design  in  order  to  address  its  hypotheses.  That  is,  an  initial  analogy  influences 
judgment  and  subsequent  elaboration  and  presented  reasons  does  not  eliminate  its 
effects. 

This  study  can  be  viewed  as  a more  conservative  test  of  analogy  effects  for 
several  reasons.  First,  the  analogy  did  not  include  an  explicit  numerical  anchor.  (It  is 
difficult,  if  not  impossible,  to  completely  eliminate  anchoring  effects  inasmuch  as 
analogies  can  be  viewed  as  sources  of  explicit  or  implicit  anchors.  Of  course,  it  also 
can  be  argued  that  anchoring  is  not  an  explanation  at  all,  but  merely  a description  of 
the  effect.)  With  this  study  design,  the  effect  of  the  analogy  was  examined  through 
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comparison  to  a control  group  for  which  no  analogy  is  provided  (again,  a more 
conservative  test).  The  pro/con  generation  task  of  the  previous  pilot  was  replaced 
with  pro/con  presentation.  Presentation  of  evenhanded  evidence  can  be  viewed  as  a 
stronger  attempt  to  neutralize  analogy  effects  because  it  reduces  potential  output 
interference  effects  that  inhibit  the  generation  of  non-supportive  evidence  by  subjects. 

In  addition,  this  study  included  another  opportunity  for  analytic  thinking 
through  the  use  of  a decomposition  task.  A more  complete  discussion  of 
decomposition  is  reserved  for  study  4.  For  now,  readers  may  view  the  decomposition 
task  as  simply  another  proposed  method  for  inducing  analytic  thinking  by  prompting 
subjects  to  think  about  key  drivers  of  new  product  success.  Subjects  were  instructed 
to  use  a computational  algorithm  to  estimate  market  share  by  combining  their 
estimates  for  awareness,  trial,  liking  and  switching.  Some  subjects  were  asked  to 
further  elaborate  on  one  component  of  the  decompositional  task.  As  before,  I 
hypothesize  that  analytic  thinking  (in  this  case  prompted  by  the  decomposition  task) 
fails  to  eliminate  the  effect  of  an  initial  analogy  on  judgment. 

Method 

Subjects  and  Design 

Subjects  were  undergraduate  students  who  participated  for  extra  credit  in  an 
introductory  marketing  class.  Each  subject  was  randomly  assigned  to  one  of  12  cells 
in  a 3 (Type  of  elaboration:  scenario,  reasons,  scenario  + reasons)  x 2 (Analogy: 
positive/none)  x 2 (Type  of  decomposition)  between-subjects  design,  with  judgments 
taken  at  three  points  in  time  (after  initial  elaboration,  after  presented  evidence,  and 


90 


after  decomposition).  A total  of  101  subjects  participated,  with  the  number  in  each 
cell  ranging  from  7 to  11.  (This  small  cell  size  is  offset  by  planned  analyses  that 
collapse  across  selected  factors.) 

Materials  and  Procedure 


All  experimental  materials  took  the  form  of  surveys  distributed  to  each 
subject.  Subjects  worked  through  the  materials  at  their  own  pace.  The  survey  asked 
subjects  to  assume  the  role  of  a manager  at  Procter  & Gamble  and  think  about  the 
future  of  olestra  (a  fat-replacement  food  additive).  Subjects  were  given  some 
background  information  on  olestra.  Some  subjects  were  cued  to  think  about  an 
analogous  product  (aspartame);  others  were  not.  The  aspartame  cue  was  intended  to 
provide  a positive  analogy  for  subjects  thinking  about  olestra.  Specifically,  subjects 
in  the  analogy  conditions  were  told: 

You  have  just  read  a report  that  gives  you  some  background  information  about 
olestra.  Now,  please  take  a few  moments  to  think  about  the  future  of  olestra. 
As  you  think  about  olestra,  you  may  find  it  helpful  to  consider  other  food 
additives  in  the  marketplace,  such  as  aspartame  (better  known  by  its  brand 
name  NutraSweet). 

Subjects  in  the  no-analogy  condition  were  told: 

You  have  just  read  a report  that  gives  you  some  background  information  about 
olestra.  Now,  please  take  a few  moments  to  think  about  the  future  of  olestra. 

All  subjects  were  then  instructed  to  elaborate  on  the  future  of  this  new  product  by 

generating  a scenario,  generating  reasons,  or  generating  a scenario  followed  by 

reasons.  The  scenario  and  reasons  instructions  were  designed  to  be  neutral.  In  the 

scenario  conditions,  subjects  were  told: 
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We  are  particularly  interested  in  what  you  think  the  future  will  be  like  for  this 
new  product.  What  will  olestra’s  future  be  like?  How  will  it  unfold?  What 
events  will  occur?  How  will  things  turn  out  for  olestra?  Please  think  about  a 
scenario  that  describes  olestra’s  future.  (A  scenario  is  a story  that  describes 
how  events  unfold  over  time  and  includes  vivid  details  that  create  a mental 
picture  or  vision  of  the  future.) 

In  the  reasons  conditions,  subjects  were  told: 

We  are  particularly  interested  in  what  you  think  the  future  will  be  like  for  this 
new  product.  What  will  olestra’s  future  be  like?  Why?  What  factors  will 
influence  olestra’s  future?  Please  think  about  reasons  for  and  against  the 
success  of  this  new  product. 

In  the  scenario  + reasons  conditions,  subjects  received  both  instructions. 

All  subjects  then  predicted  the  future  of  olestra  by  responding  to  a series  of 
questions,  including  binary  choice  measures  and  confidence,  success  ratings,  and 
adoption  estimates  over  time.  The  first  booklet  was  then  removed,  and  subjects  were 
given  a second  booklet  with  the  following  instructions.  “Now,  please  take  a few 
minutes  to  consider  another  opinion  about  olestra’s  future.  You  may  or  may  not  find 
this  information  helpful  in  forming  your  own  professional  opinion.”  The  information 
consisted  of  eight  “possible  reasons  for  olestra’s  success”  and  eight  “possible  reasons 
for  olestra’s  failure,”  designed  to  provided  relatively  balanced  and  mixed  evidence. 
Subjects  repeated  their  judgments  of  olestra’s  future  and  rated  the  information  and  its 
impact  on  their  judgment. 

Following  the  presented  evidence,  subjects  were  asked  to  use  either  a simple 

or  elaborate  “break-down  exercise”  to  aid  their  judgment.  Subjects  were  told: 

Becoming  a regular  user  consists  of  4 steps:  awareness,  trial,  liking  and 
switching.  To  become  a regular  user  of  olestra,  a consumer  must  pass  through 
all  4 of  these  steps. 
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In  the  elaborate  decomposition  task  only,  subjects  read  additional  information 
explaining  that  “it  is  also  possible  to  break  down  these  steps  into  sub-steps...  Sub- 
steps can  include  buyer  and/or  seller  activities  or  other  events.”  These  subjects  were 
asked  to  choose  one  of  the  four  steps  and  break  it  down  into  sub-steps  as  an  exercise. 
All  subjects  then  carried  out  the  decomposition  task,  by  estimating  the  “percentage  of 
people  who  become  aware  of  olestra,”  “percentage  of  'aware’  people  who  decide  to 
try  olestra,”  “percentage  of ‘triers’  who  like  olestra,”  and  “percentage  of  Tikers’  who 
switch  to  using  olestra”.  Subjects  were  instructed  on  how  to  combine  these  numbers 
through  multiplication  to  arrive  at  an  estimate  for  “percentage  regular  users”. 

Subjects  were  then  told  that  “You  may  or  may  not  wish  to  make  adjustments  to  this 
number  based  on  your  own  judgment”  before  providing  a final  estimate  of  % regular 
users  in  2010.  Subjects  then  indicated  their  confidence  in  this  estimate.  Finally,  in 
the  last  set  of  questions,  subjects  rated  their  agreement  with  a series  of  statements 
concerning  olestra  and  indicated  their  personal  opinions  about  olestra,  aspartame 
(NutraSweet),  and  their  experience  in  the  survey. 

Results  and  Discussion 

This  study  investigates  whether  an  analogy  influences  judgment  in  a new 
product  forecasting  task  and  whether  elaboration  can  neutralize  the  effect  of  an  initial 
analogy.  A main  effect  of  the  analogy  on  judgment  would  indicate  that  the  analogy 
induced  bias.  Depending  on  when  the  dependent  measure  occurs,  it  is  also  possible  to 
test  for  the  effectiveness  of  elaboration  (at  time  1 ),  presented  evidence  (at  time  2),  and 
decomposition  (at  time  3)  in  correcting  for  this  bias. 


93 


Judgment  After  Elaboration  and  Presented  Evidence 

Results  on  most  dependent  variables  support  the  hypothesis  that  the  effect  of 
an  initial  analogy  will  endure  despite  elaboration  and  presented  evidence.""  Subjects 
were  asked  to  rate  whether  olestra  is  likely  to  succeed  and  how  well  it  will  sell  (on  1- 
7 scales)  both  after  initial  elaboration  and  after  presented  evidence  (see  Table  19  and 
Figure  9).  These  attitude  ratings  were  analyzed  in  a MANOVA  with  analogy  and 
type  of  elaboration  as  between-subjects  factors  and  repeated  judgment  as  a within- 
subject  factor.  For  judgments  of  how  well  olestra  will  sell,  subjects  who  were  given 
an  initial  positive  analogy  were  more  optimistic  than  subjects  who  were  not  given  an 
analogy  both  after  initial  elaboration  (Fi9i=3.85,  p=.053)  and  after  presented 
evidence  (Fit9i=7.70,  p=.007).  The  change  in  this  rating  from  time  1 to  time  2 was 
influenced  by  the  type  of  elaboration  (F  1,91— 4. 1 5,  p=.0 1 9);  nonetheless,  the  change 
failed  to  eliminate  the  effect  of  the  initial  analogy  on  final  judgments."  Although 
there  were  no  significant  effects  of  the  analogy  on  judgments  of  whether  olestra  is 
likely  to  succeed  after  elaboration  or  presented  evidence  (both  F<1),  the  change  in  this 
rating  from  time  1 to  time  2 was  marginally  more  positive  when  subjects  were  given 
the  positive  analogy  (F i,9i=3 . 1 6,  p=.079). 
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CATMOD  analysis  of  a binary  judgment  of  success/failure  taken  after  initial  elaboration  revealed 
no  significant  effects  of  analogy  or  type  of  elaboration  on  judgment.  Neither  did  an  ANOVA  on 
confidence  in  this  binary  judgment  (all  non-significant).  Notably,  the  majority  of  subjects  (77  out  of 
98,  or  78.6%)  predicted  success  for  olestra  (77  out  of  98,  or  78.6%).  The  lack  of  significant  findings 
on  these  dependent  variables  could  be  due  to  a ceiling  effect  because  subjects  are  quite  favorable 
toward  olestra  even  in  the  no-analogy  condition.  These  results  are  reported  for  completeness  in  Table 
18. 

23  Because  the  type  of  initial  elaboration  (i.e.,  scenarios  vs.  reasons)  does  not  appear  to  have  consistent 
and  interpretable  effects  on  judgment  across  the  dependent  variables,  this  factor  will  not  be  discussed 
further. 
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Similarly,  subjects  estimated  the  percentage  of  regular  users  of  olestra  over 
time  both  after  initial  elaboration  and  after  presented  evidence  (see  Table  20).  A 
MANOVA  of  these  estimates  with  analogy  and  type  of  elaboration  as  between- 
subjects  factors  and  estimation  period  and  repeated  judgment  as  wi thin-subject  factors 
revealed  a significant  main  effect  of  the  analogy  on  all  of  these  estimates  (F  1,93=6.40, 
p=.01 3). 24  Averaging  across  estimation  periods,  judgments  were  higher  when 
subjects  were  given  an  analogy  (versus  not)  both  after  elaboration  (F  1.92=6.71 , 
p=.01 1)  and  after  presented  evidence  (Fp92=5.42,  p=.022).  The  change  in  judgments 
from  time  1 to  time  2 were  influenced  by  an  interaction  of  the  analogy  and  type  of 
elaboration  (F2,93=2.90,  p=. 060). 2:1  Nonetheless,  this  change  failed  to  eliminate  the 
effect  of  the  initial  analogy  on  judgment  (see  Figure  10). 

Overall,  these  results  on  key  dependent  variables  (success  ratings  and 
adoption  estimates,  both  after  elaboration  and  after  presented  evidence)  support  the 
hypothesis  that  the  influence  of  an  analogy  on  judgment  is  relatively  enduring.  As  a 
further  demonstration  of  this  finding,  an  alternative  analysis  was  conducted  on  these 


~4  The  MANOVA  also  revealed  interactions  of  analogy  and  type  of  elaboration  with  estimation  period 
(quadratic  interaction:  F2, 93=2. 45,  p=. 092;  cubic  interaction:  F2, 93=4. 32,  p=.0 1 6).  These 
interactions  reflect  the  notion  that  the  effect  of  the  analogy  depends  upon  the  type  of  elaboration  and 
the  time  period.  There  is  some  indication  that  that  the  effect  of  the  analogy  is  greater  for  scenarios 
than  analogies  but  it  does  not  appear  to  hold  consistently  across  time  periods.  The  possibility  that 
scenarios  exacerbate  analogy-driven  effects  or  that  scenarios  and  analogies  are  mutually  reinforcing  is 
intriguing  and  merits  further  investigation. 

25  The  interaction  pattern  is  consistent  but  difficult  to  interpret.  With  no  analogy,  change  is  most 
negative  in  the  reasons  condition  and  least  negative  in  the  scenario+reasons  condition.  With  an 
analogy,  change  is  most  negative  in  the  scenario+reasons  condition  and  least  negative  in  the  scenario 
condition.  Again,  an  investigation  of  the  interaction  of  type  of  elaboration  with  analogy  merits  future 
research. 
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dependent  variables.26  First,  a principal  components  factor  analysis  was  performed, 
and  three  factors  with  eigenvalues  greater  than  one  emerged.  Adoption  estimates  for 
the  first  two  estimation  periods  loaded  on  the  first  factor  (called  ‘early  success’), 
success  ratings  loaded  on  the  second  factor  (called  ‘overall  success’),  and  adoption 
estimates  for  the  last  two  estimation  periods  loaded  on  the  third  factor  (called 
‘ultimate  success’).  These  factor  scores  for  each  subject  were  then  submitted  to  a 
MANOVA  with  analogy  and  type  of  elaboration  as  independent  variables.  The  main 
effect  of  analogy  was  significant  (Fi>9i=5.77,  p=. 01 8):  factor  scores  were  higher  for  a 
positive  (versus  no)  analogy,  indicating  more  optimistic  judgments  by  subjects 
presented  with  an  analogy.  Similarly,  when  a principal  components  factor  analysis 
was  restricted  to  one  factor  only,  an  ANOVA  of  this  factor  score  for  each  subject 
revealed  a significant  main  effect  of  analogy  (F],9o=5.54,  p=.021).  Thus,  this 
alternative  analysis  also  demonstrates  a consistent  effect  of  analogy  that  endures 
despite  elaboration  (scenario  and/or  reason  generation)  and  presented  evidence. 

Overall,  these  results  suggest  that  elaboration  (via  a scenario  and  generated 
and  presented  reasons)  fails  to  eliminate  the  effect  of  an  initial  analogy.  Judgments 
continued  to  be  influenced  by  the  initial  analogy  after  elaboration  and  presentation  of 
evidence.  Further  evidence  of  the  effects  of  the  analogy  on  judgment  can  be  found  in 
subject  ratings  of  the  presented  reasons  (see  Table  21).  While  positive  ratings  did  not 
differ  (F<1),  negative  ratings  were  marginally  affected  by  an  interaction  of  the 


26  The  details  of  these  factor  analyses  are  omitted  due  to  space  limitations. 
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analogy  and  type  of  elaboration  (F2,92=2.56,  p=  0829). 27  Importantly,  subjects  given 
an  initial  analogy  reported  that  the  presented  evidence  made  them  more  favorable  and 
more  confident  about  olestra’s  success  compared  to  subjects  given  no  analogy 
(Fi.92-3.58,  p=.062).  These  self-report  measures  of  attitude  change  (see  Table  21)  are 
indicative  of  subjects’  mindsets  and  provide  another  indication  of  the  continued 
influence  of  the  analogy  on  judgment. 

Judument  after  Decomposition 

To  analyze  the  effect  of  decomposition,  I collapsed  across  the  type  of 
elaboration  to  avoid  instability  induced  by  small  cell  sizes."8  The  influence  of  the 
analogy  is  evident  in  subjects’  estimates  in  the  decomposition  task.  Recall  that  this 
decomposition  task  asked  subjects  to  estimate  the  percentage  of  regular  users  of 
olestra  in  2010  by  breaking  it  down  into  component  estimates  for  awareness,  trial, 
liking  and  switching.  A MANOVA  of  the  four  component  estimates  as  a function  of 
the  analogy  and  type  of  decomposition  reveals  a significant  overall  effect  of  the 
analogy  (Fi  9s=3.75,  p=.  056).  On  average,  component  estimates  are  higher  for 
subjects  given  an  initial  analogy,  indicating  that  component  estimates  are  susceptible 
to  the  analogy  during  decomposition  (see  Table  22). 

Surprisingly,  the  effect  of  the  analogy  did  not  carry  over  to  the  calculation 
made  by  subjects  based  on  these  component  estimates,  although  the  effect  of  the 

27  However,  the  pattern  of  results  does  not  seem  easily  interpretable. 

28  An  additional  analysis  was  conducted  without  collapsing  across  the  factor  representing  type  of 
elaboration.  The  pattern  of  results  does  not  change  significantly,  although  some  higher-order 
interactions  do  occur.  Because  of  the  small  cell  sizes,  the  stability  of  these  interactions  seems 
questionable. 
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analogy  is  in  the  expected  direction  (F i,92=l -27,  p=.263).  However,  a combined 
estimate  calculated  by  computer  does  reveal  the  expected  effect  of  the  analogy 
(Fi  92=3.03,  p=.085).  Subjects’  own  calculations  systematically  erred  in  a positive 
direction,  perhaps  reflecting  difficulty  calculating  small  percentages  or  understanding 
conjunctive  probabilities  (or  reflecting  general  optimism  toward  the  new  product). 
Interestingly,  when  subjects  were  given  the  opportunity  to  adjust  their  component- 
based  estimate  based  on  their  own  judgment,  they  did  so,  again  in  a systematically 
positive  direction.  Furthermore,  their  adjustments  were  influenced  by  the  initial 
analogy  and  were  more  positive  for  subjects  given  an  initial  positive  analogy 
(Fi;92=3.39,  p=.069).  Thus,  the  decomposition  task  proved  to  be  ineffective  at 
eliminating  the  effect  of  the  initial  analogy  and  was  itself  susceptible  to  the  analogy’s 
effects.29 

As  further  evidence  of  the  persistent  effect  of  the  initial  analogy  and  the 
failure  of  elaboration,  presented  evidence,  and  decomposition  to  eliminate  its  effects, 
I turn  now  to  MANOVAs  of  some  final  attitudinal  measures.  Subjects  were  asked  to 
rate  their  agreement  with  items  regarding  the  success  of  olestra  (see  Table  23). 


29  The  effect  of  the  decomposition  task  can  also  be  tested  by  examining  the  change  in  judgment  for  the 
dependent  variable  measuring  percentage  regular  users.  Recall  that  subjects  also  estimated  the 
percentage  of  regular  users  of  olestra  in  2010  after  initial  elaboration  and  after  presented  evidence  (as 
the  fourth  estimation  period  in  the  adoption  curve).  A within-subject  analysis  of  this  repeated  measure 
revealed  an  overall  main  effect  of  analogy  (F|  92=7.04,  p=.0094),  qualified  by  an  interaction  with  the 
type  of  decomposition  (F,  92=4.97,  p=.028).  With  no  analogy,  the  elaborate  (versus  simple)  form  of 
decomposition  appeared  to  induce  greater  change  in  judgment;  with  an  analogy,  the  change  was 
smaller.  Thus,  the  decomposition  task  seemed  to  have  greater  impact  in  the  absence  of  an  analogy, 
perhaps  because  the  breakdown  task  is  itself  susceptible  to  the  initial  analogy,  thereby  reducing  its 
effectiveness.  This  possibility  merits  consideration  and  a further  discussion  of  decomposition  as  a 
debiasing  technique  is  reserved  for  Study  4.  Nonetheless,  as  indicated  by  the  overall  main  effect  of 
analogy  on  these  estimates,  the  change  induced  by  the  decomposition  task  failed  to  eliminate  analogy- 
induced  effects  on  judgment. 
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Subjects  who  were  given  an  initial  analogy  were  more  likely  to  agree  with  these  items 
than  subjects  not  given  an  analogy  (Fig9=3.58,  p=.062).  These  results  can  be 
interpreted  as  providing  evidence  of  the  analogy’s  effect  on  elaboration  and  inference 
making;  that  is,  subjects  appear  to  be  reasoning  by  analogy  to  aspartame  in  thinking 
about  the  future  of  olestra. 

Thus,  the  overall  evidence  from  this  study  suggests  that  an  analogy  can  have 
persistent  effects  on  judgment.  Under  relatively  conservative  test  conditions,  neither 
elaboration  (via  scenario  or  reasons)  nor  presented  evidence  nor  decomposition  was 
sufficient  to  eliminate  the  effect  of  an  initial  analogy  on  judgment.  The  analogy 
continued  to  dominate  judgment  despite  several  attempts  to  induce  evenhanded 
analytic  thinking. 


Table  18. 


Pilot  study  4:  predictions  of  success  and  confidence  in  olestra  as  a function  of 
analogy  and  type  of  elaboration. 


Analogy 

Type  of  Elaboration 

# of  Success 
Predictions 

Confidence  in  Success 

None 

Reasons 

14/18 

68.33  (22.75) 

None 

Scenario  + Reasons 

10/  16 

63.13  (28.69) 

None 

Scenario 

12/15 

75.71  (19.89) 

Positive 

Reasons 

16/18 

73.06  (21.84) 

Positive 

Scenario  + Reasons 

12/15 

70.67  (27.05) 

Positive 

Scenario 

13  / 17 

71.18(25.71) 

Note:  Overall,  78%  (77  out  of  99)  of  predictions  were  for  new  product  success. 
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Table  19. 


Pilot  study  4:  attitude  ratings  toward  olestra  after  elaboration  and  after  presented 

evidence. 


Analogy 

Type  of  Elaboration 

N 

Attitude  Rating 
(Unlikely/Likely  to  Succeed) 
After  After  Change 

elaboration  evidence 

None 

Reasons 

17 

4.94  (1.09) 

4.71  (1.05) 

-0.24  (0.44) 

None 

Scenario  + Reasons 

15 

5.00  (1.60) 

4.60(1.59) 

-0.40  (0.74) 

None 

Scenario 

14 

5.42  (1.34) 

4.93  (1.59) 

-0.50  (0.76) 

Positive 

Reasons 

19 

5.16(1.42) 

5.00(1.45) 

-0.16(0.69) 

Positive 

Scenario  + Reasons 

15 

5.00(1.56) 

4.93  (1.49) 

-0.07(1.16) 

Positive 

Scenario 

17 

4.88  (1.62) 

4.88(1.41) 

-0.00(1.06) 

Main  effect  of  analogy: 

n.s. 

n.s. 

sig. 

Attitude  Rating 

(Will  Sell  Poorly  / Sell  Well) 

After 

After 

Change 

elaboration 

evidence 

None 

Reasons 

17 

4.82  (1.29) 

4.59(1.12) 

-0.24  (0.56) 

None 

Scenario  + Reasons 

15 

5.00  (1.36) 

4.67(1.40) 

-0.33  (0.62) 

None 

Scenario 

14 

5.57  (1.45) 

4.64(1.15) 

-0.93  (0.73) 

Positive 

Reasons 

19 

5.42  (1.07) 

5.32  (1.00) 

-0.11  (0.57) 

Positive 

Scenario  + Reasons 

15 

5.80  (0.86) 

5.47(1.13) 

-0.33  (1.18) 

Positive 

Scenario 

17 

5.65  (1.27) 

5.12(1.27) 

-0.53  (1.01) 

Main  effect  of  analogy: 

sig. 

sig. 

n.s. 
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Figure  9. 

Pilot  study  4:  attitude  ratings  (will  sell  very  poorly/well)  as  a function  of  analogy 
after  elaboration  and  after  presented  evidence. 
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Table  20. 

Pilot  study  4:  adoption  estimates  for  olestra  after  elaboration  and  after  presented 

evidence. 


Analogy  Type  of  N Adoption  Estimates 

Elaboration  (after  elaboration) 


at  1 
year 

at  2 
years 

at  5 
years 

at  10 
years 

Average 

None 

Reasons 

18 

39.06 

44.53 

52.22 

56.33 

48.03 

(27.85) 

(24.20) 

(25.30) 

(31.11) 

(23.27) 

None 

Scenario  + 

16 

38.25 

39.69 

38.31 

36.31 

38.14 

Reasons 

(25.25) 

(18.21) 

(23.13) 

(33.07) 

(17.66) 

None 

Scenario 

14 

31.14 

34.50 

46.50 

52.64 

41.20 

(27.79) 

(20.36) 

(17.17) 

(27.11) 

(13.57) 

Positive 

Reasons 

19 

41.53 

47.16 

54.11 

64.42 

52.60 

(28.73) 

(23.15) 

(20.80) 

(28.06) 

(19.15) 

Positive 

Scenario  + 

15 

39.00 

45.27 

55.00 

63.67 

50.73 

Reasons 

(29.53) 

(23.15) 

(17.79) 

(23.10) 

(16.79) 

Positive 

Scenario 

17 

50.00 

56.59 

54.82 

54.65 

54.01 

(27.10) 

(23.33) 

(26.22) 

(37.63) 

(20.55) 

Effect  of  Analogy 

n.s. 

sig. 

sig. 

sig. 

sig. 

Adoption  Estimates 


(after  presented  evidence) 


at  1 
year 

at  2 
years 

at  5 
years 

at  10 
years 

Average 

None 

Reasons 

18 

34.83 

37.42 

43.72 

48.50 

41.12 

(28.07) 

(24.52) 

(25.18) 

(29.24) 

(23.71) 

None 

Scenario  + 

16 

38.25 

39.81 

38.69 

38.19 

38.73 

Reasons 

(26.54) 

(22.13) 

(23.80) 

(32.01) 

(21.90) 

None 

Scenario 

14 

28.86 

29.93 

35.64 

43.57 

34.50 

(24.65) 

(20.36) 

(18.50) 

(26.32) 

(15.72) 

Positive 

Reasons 

19 

37.05 

42.47 

49.37 

58.95 

47.69 

(29.25) 

(23.00) 

(23.68) 

(30.03) 

(21.15) 

Positive 

Scenario  + 

15 

33.33 

37.93 

47.43 

52.03 

42.68 

Reasons 

(28.70) 

(23.45) 

(19.68) 

(28.46) 

(19.06) 

Positive 

Scenario 

17 

50.29 

54.71 

54.82 

52.76 

53.15 

(28.80) 

(22.11) 

(24.24) 

(33.70) 

(19.94) 

Effect  of  Analogy 

n.s. 

sig.* 

sig. 

sig. 

sig. 

Note:  * indicates  that  analogy  X type  of  elaboration  was  also  significant. 
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Figure  10. 

Pilot  study  4:  average  adoption  estimate  as  a function  of  analogy  after  elaboration 

and  after  presented  evidence. 

Table  21. 


Pilot  study  4:  ratings  of  presented  evidence  and  change  induced  by  evidence  (self- 
reported). 


Analogy 

Type  of 
Elaboration 

N 

Ratings 

of 

Positive 

Reasons 

Ratings 

of 

Negative 

Reasons 

Change  in 
Attitude 
(Favorability 
toward 
Success) 

Change  in 
Attitude 
(Confidence 
in  Success) 

None 

Reasons 

16 

10.39 

10.61 

4.75 

4.63 

(1.85) 

(2.52) 

(0.93) 

(1.09) 

None 

Scenario  + 

16 

10.00 

8.75 

5.13 

5.06 

Reasons 

(1.83) 

(3.11) 

(0.72) 

(0.85) 

None 

Scenario 

15 

10.86 

9.64 

4.80 

4.87 

(2.11) 

(2.59) 

(1.47) 

(1.60) 

Positive 

Reasons 

19 

10.68 

8.89 

5.37 

5.42 

(2.03) 

(2.79) 

(1.30) 

(1.17) 

Positive 

Scenario  + 

15 

10.71 

9.21 

5.73 

5.53 

Reasons 

(2.40) 

(2.89) 

(1.44) 

(1.51) 

Positive 

Scenario 

17 

10.65 

10.76 

4.94 

5.00 

(2.37) 

(2.41) 

(1.48) 

(1.50) 
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Table  22. 


Pilot  study  4:  decomposition  task  estimates  as  a function  of  analogy  and  type  of 
decomposition. 


Analogy 

Type  of 

N 

% aware- 

% trial  % liking 

% switch 

Decomposition 

ness 

None 

Simple 

25 

76.40 

58.20  47.28 

41.64 

(17.18) 

(20.51)  (21.57) 

(24.98) 

None 

Elaborate 

24 

72.71 

55.42  38.96 

35.17 

(23.47) 

(19.61)  (17.00) 

(20.37) 

Positive 

Simple 

21 

77.05 

61.76  54.57 

47.19 

(19.66) 

(20.10)  (20.85) 

(21.47) 

Positive 

Elaborate 

29 

75.72 

60.79  56.10 

46.93 

(18.48) 

(20.67)  (22.63) 

(22.22) 

Algorithm 

Estimate 

Estimate 

output 

calculated  by 

adjusted  by 

subjects 

subjects 

None 

Simple 

24 

14.78 

20.38 

30.05 

(17.98) 

(22.22) 

(21.29) 

None 

Elaborate 

24 

9.14 

18.12 

26.77 

(11.89) 

(19.31) 

(17.91) 

Positive 

Simple 

21 

17.35 

23.86 

39.46 

(17.59) 

(21.16) 

(24.29) 

Positive 

Elaborate 

27 

17.35 

24.30 

44.52 

(12.63) 

(20.91) 

(25.04) 

Note:  Algorithm  output  is  the  product  of  the  four  component  estimates. 
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Table  23. 


Pilot  study  4:  agreement  with  statements  favoring  olestra  as  a function  of  analogy. 


No  Analogy 

Analogy 

P&G  should  prominently  display  the  Olean  brand 
name  on  any  foods  that  contain  its  brand  of 
olestra. 

5.41  (1.54) 

5.85  (1.20) 

Eventually,  olestra  will  become  as  common  as 
aspartame  (better  known  by  its  brand  name 
Nutrasweet). 

4.76  (1.40) 

5.26  (1.33) 

I found  it  difficult  to  understand  how  olestra  works 
in  the  human  body,  (omitted) 

— “ 

Olestra  has  many  benefits. 

4.91  (1.43) 

5.00(1.38) 

Olestra  has  many  risks,  (reverse-coded) 

3.50(1.46) 

3.47  (1.52) 

The  benefits  of  olestra  outweigh  the  risks. 

4.28  (1.38) 

4.49(1.44) 

Comparing  olestra  to  aspartame  is  (not)  appropriate 
in  judging  olestra’s  future,  (reverse-coded) 

3.61  (1.68) 

4.19(1.60) 

The  future  of  olestra  will  be  as  successful  as 
aspartame. 

3.96(1.58) 

4.51  (1.20) 

Average 

4.35  (0.90) 

4.68  (0.78) 

CHAPTER  10 

STUDY  3:  ANALOGIES  AND  COUNTER- ANALOGIES 


The  pilot  studies  suggest  that  analogies  have  persistent  effects  on  judgment. 
These  pilots  also  indicate  that  elaboration  (for  example,  generating  evenhanded 
reasons)  failed  to  eliminate  the  effects  of  an  initial  analogy  on  judgment.  In  light  of 
these  persistent  effects  of  analogies,  this  study  strengthens  debiasing  efforts  by 
investigating  whether  counterexplanation  and  counter- analogies  are  sufficient  to 
neutralize  the  effect  of  an  initial  analogy  on  judgment.  It  does  so  in  a way  that 
permits  us  to  test  whether  analogies  are  differentially  persistent  to  debiasing  with 
counterexplanation,  compared  to  one-sided  explanation.  Moreover,  this  study  also 
offers  direct  evidence  of  “bias”  by  investigating  whether  consideration  of  multiple 
analogies  is  susceptible  to  order  effects.  In  fulfilling  these  two  objectives,  this  study 
will  provide  evidence  regarding  the  potential  effectiveness  of  multiple  analogies  as  a 
management  tool  for  improving  forecasting.  (Recall  that,  in  a similar  fashion,  Study 
1 examined  the  use  of  multiple  scenarios  in  judgmental  forecasting.) 

Of  course,  order  effects  in  reasoning  are  not  new.  Indeed,  I have  already 
reviewed  prior  research  that  demonstrates  order  effects  in  generating  pro  and  con 
reasons  (e.g.,  Hoch  1984).  However,  the  results  from  the  pilot  studies  suggest  that 
analogies  may  dominate  reasons.  If  so,  counterexplanation  may  be  less  effective 
when  it  follows  a one-sided  analogy  rather  than  one-sided  reasons.  Once  a schema 
has  been  elaborated  in  support  of  an  initial  analogy,  it  may  be  difficult  to  undo  this 
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schema  completely  and  then  evenhandedly  consider  the  opposite  point  of  view. 

Based  on  this  line  of  reasoning  and  my  prior  evidence,  I predict  that 
counterexplanation  will  be  less  effective  at  neutralizing  the  effect  of  an  initial  analogy 
(compared  to  initial  reasons).  In  other  words,  analogies  will  be  more  resistant  to 
debiasing  with  counterexplanation  than  reasons,  thereby  leading  to  stronger  primacy 
effects  in  judgment. 

The  second  objective  of  this  study  is  to  investigate  the  effectiveness  of 
counter-analogies  in  eliminating  the  effects  of  an  initial  analogy.  Interestingly, 
Einhorn  and  Hogarth  (1987)  recommend  the  use  of  alternative  analogies  to  improve 
judgment.  “Each  metaphor  illustrates  a different  factor,  an  alternative  way  of 
thinking.  No  metaphor  by  itself  is  adequate;  considering  them  all  provides  a more 
complete  picture”  (p.68).30  As  an  example,  the  authors  consider  how  various 
metaphors  illuminate  different  dimensions  of  a business  school:  finishing  school 
(students  mature  before  entering  corporate  life),  military  academy  (preparing  for 
economic  warfare),  monastery  (receiving  indoctrination  in  economic  theology), 
diploma  mill,  job  shop,  etc.  In  other  words,  reasoning  by  analogy  may  unduly 
emphasize  the  importance  of  some  dimensions  of  judgment  at  the  expense  of  others. 

In  contrast,  an  alternative  analogy  may  break  the  conditional  frame  of  reference 
established  by  the  initial  analogy  (consistent  with  Koehler  1 994)  and  re-frame  and 
highlight  different  aspects  of  the  judgment  task.  Hence,  prompting  people  to  consider 


30  Einhorn  and  Hogarth  (1987)  actually  refer  to  metaphors.  A metaphor  suggests  likeness  between  two 
objects  directly  so  the  authors’  logic  seems  to  extend  to  analogies. 
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multiple  analogies  may  induce  more  evenhanded  consideration  of  the  relevant 
dimensions  (a  characteristic  of  analytic  thinking).  In  addition,  and  consistent  with  the 
line  of  argument  used  by  proponents  of  multiple  explanation  (Hirt  & Markman  1995) 
and  scenario  analysis  (Schoemaker  1993),  considering  alternative  analogies  may  be 
sufficient  to  debias  judgment. 

Holyoak  and  Thagard  (1995b)  propose  '‘analogy  therapy”  as  “an  antidote  to 
fixation”  but  comment  that  the  use  of  multiple  analogies  is  not  very  well  understood. 

I am  unaware  of  any  prior  research  that  has  investigated  the  use  of  multiple  analogies 
to  improve  judgment.  Research  on  reasoning  by  analogy  suggests  that  people  try  to 
generate  mappings  that  satisfy  certain  constraints  (i.e.,  parallel  constraint  satisfaction) 
(Spellman  & Holyoak  1992).  A similar  tendency  may  exist  when  people  consider 
multiple  analogies,  that  is,  they  may  prefer  the  analogy  that  best  fits  the  target. 
Holyoak  and  Simon  (1999)  suggest  that  both  intra-analogy  competition  and  inter- 
analogy competition  may  be  similarly  governed  by  constraint  satisfaction.  When  the 
two  analogies  are  approximately  equal  in  terms  of  structural  and  surface  similarity, 
people  may  prefer  the  analogy  that  best  supports  their  goals  (Spellman  & Holyoak 
1996).  Such  favoritism  was  evident  in  Holyoak  and  Simon’s  research  on  jury 
decision-making.  Subjects  who  reached  a verdict  in  a case  shifted  their  assessments 
in  favor  of  the  analogy  consistent  with  their  verdict.  Interestingly,  the  favored 
analogy  continued  to  influence  judgments  in  a subsequent  case.  This  result  was 
found  when  multiple  analogies  were  presented  simultaneously.  Ordered  presentation 
of  analogies  may  exacerbate  the  effect  if  earlier  reasoning  by  analogy  and  analogy- 
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driven  inference  making  serves  to  reinforce  the  initial  analogy  and  make  it  more 
difficult  for  people  to  consider  both  analogies  evenhandedly.  Anecdotal  evidence 
suggests  that  initial  analogies  can  interfere  with  the  acceptance  of  new  analogies.  For 
example,  students  typically  first  learn  the  Rutherford-Bohr  model  of  the  atom  that 
uses  an  analogy  to  the  solar  system,  but  this  orbiting  concept  conflicts  with 
subsequent  learning  of  models  of  the  atom  that  incorporate  quantum  mechanics  (see 
Holyoak  & Thagard  1995b  for  an  interesting  discussion  of  analogies  in  science). 

Thus,  the  use  of  multiple  analogies  in  judgment  is  an  open  question.  On  the 
one  hand,  alternative  analogies  may  break  the  conditional  frame  of  reference  induced 
by  the  initial  analogy  and  prompt  more  evenhanded  consideration  of  other  dimensions 
of  the  problem.  On  the  other  hand,  reasoning  by  analogy  may  drive  elaboration  that 
reinforces  an  initial  analogy’s  effects.  Once  a schema  has  been  elaborated  in  support 
of  an  initial  analogy,  it  may  be  difficult  to  undo  this  schema  completely  and 
evenhandedly  consider  an  alternative  analogy.  Based  on  this  line  of  reasoning  and 
my  prior  evidence,  I predict  that  multiple  analogies  will  be  susceptible  to  order 
effects.  That  is,  predictions  will  be  biased  in  the  direction  of  the  initial  analogy 
considered  (primacy  order  effect).  Considering  counter-analogies  will  not  debias 
judgment.  Order  effects  would  constitute  direct  evidence  of  “bias”  and  undermine 
the  effectiveness  of  multiple  analogies  as  an  aid  to  judgmental  forecasting. 
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Method 


Subjects  and  Design 

Subjects  were  undergraduate  students  who  participated  for  extra  credit  in  an 
introductory  marketing  class.  Each  subject  was  randomly  assigned  to  one  of  six  cells 
in  a 3 (Type  of  elaboration:  analogy  then  counter-analogy,  analogy  then  counter- 
reasons, reasons  then  counter-reasons)  x 2 (Order:  success  then  failure  versus  failure 
then  success)  between-subjects  design.  A total  of  125  subjects  participated,  with  the 
number  in  each  cell  ranging  from  20  to  2 1 . 

Materials  and  Procedure 


All  experimental  materials  took  the  form  of  surveys  distributed  to  each 
subject.  The  survey  asked  subjects  to  play  the  role  of  a marketing  manager  and  think 
about  the  future  of  vitalia-based  snack  foods  at  SnackTime.  (Vitalia  is  a fictitious 
product  ingredient.)  Subjects  were  given  some  background  information  on  vitalia  and 
SnackTime  as  follows: 

Imagine  that  you  are  a marketing  manager.  You  worked  for  a 
number  of  years  in  the  new  product  division  at  a major  beverage 
company.  This  experience  has  helped  you  to  land  a new  job  as  head  of 
new  products  at  a snack  food  company,  SnackTime  Inc.  This  large 
company  makes  a variety  of  snack  foods,  including  chips,  cookies, 
crackers  and  so  on.  SnackTime’s  snack  food  products  are  very 
popular  with  consumers.  SnackTime  is  facing  an  important  decision 
concerning  their  products  and  has  asked  for  your  views  as  new  product 
manager. 

The  Research  & Development  Division  at  SnackTime  has 
developed  a new  product,  called  vitalia™.  When  vitalia  is  added  to  the 
snack  food  formula  during  the  manufacturing  process,  snack  food 
flavors  change  and  become  stronger.  In  blind  taste  tests,  consumers 
slightly  preferred  vitalia-formula  snacks  to  regular-formula  snacks. 

The  research  department  is  proposing  that  SnackTime  switch  to  a 
vitalia  formula  for  all  of  its  snack  foods. 


109 


On  the  next  few  pages,  you  will  be  asked  to  consider 
SnackTime’s  situation  and  offer  your  views  on  whether  SnackTime 
should  offer  vitalia-formula  snack  foods. 

Subjects  then  considered  reasons  for  and  against  the  success  of  vitalia-formula  snack 
foods.  Some  subjects  also  received  an  analogy  manipulation  (indicated  in  brackets  in 
the  following  instructions).  In  their  instructions,  subjects  were  told  to  think  about 
success  as  follows: 

Think  about  reasons  why  switching  to  vitalia-formula  snack  foods  could  be  a 
success  for  SnackTime.  (As  you  think  about  reasons  for  success,  you  may 
find  it  helpful  to  consider  Coca-Cola’s  experience  with  the  success  of  Diet 
Coke.)  List  as  many  possible  reasons  as  you  can  why  SnackTime  should 
switch  to  vitalia-formula  snack  foods. 

Subjects  were  asked  to  think  about  failure  with  the  following  instructions: 

Think  about  reasons  why  switching  to  vitalia-formula  snack  foods  could  be  a 
failure  for  SnackTime.  (As  you  think  about  reasons  for  failure,  you  may  find 
it  helpful  to  consider  Coca-Cola’s  experience  with  the  failure  of  New  Coke.) 
List  as  many  possible  reasons  as  you  can  why  SnackTime  should  not  switch  to 
vitalia-formula  snack  foods. 

Both  writing  tasks  were  timed  and  lasted  three  minutes;  between  each  task,  subjects 
carried  out  a three-minute  distractor  task  (listing  five  examples  each  of  cities,  colors, 
sports,  universities,  academic  majors,  movies,  etc.)  Subjects  received  the  writing 
tasks  in  either  order  (success  then  failure  or  failure  then  success).  Some  subjects 
received  no  analogies  (reasons  then  counter-reasons);  some  subjects  received  the 
initial  analogy  only  (analogy  then  counter-reasons);  some  subjects  received  both 
analogies  (analogy  then  counter-analogy).  Subjects  then  answered  a series  of 
questions  regarding  their  recommendation  to  switch  to  vitalia-formula  snack  foods, 
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confidence  in  this  recommendation,  opinion  toward  vitalia-formula  snack  foods,  and 
estimates  of  percentage  of  Americans  consuming  this  new  product  over  time. 

Results  and  Discussion 

The  study  predicted  that  judgments  would  be  more  biased  toward  an  initial 
analogy  than  toward  initial  reasons  generated.  In  other  words,  an  analysis  of  the  2 
(analogy  versus  reasons)  x 2 (order)  design  is  expected  to  reveal  a significant  two- 
way  interaction,  with  a stronger  primacy  effect  in  the  analogy-then-counter-reasons 
cells  than  the  reasons-then-counter-reasons  cells.  In  addition,  a primacy  order  effect 
is  also  expected  in  the  analogy-then-counter-analogy  cells.  No  prediction  is  made 
regarding  its  strength  relative  to  the  other  order  effects  at  this  time.  Note  also  that  an 
order  effect  in  considering  multiple  analogies  helps  to  rule  out  demand  as  an 
alternative  explanation  for  these  results.  Results  across  the  dependent  measures  are 
fairly  consistent  with  these  predictions. 

First,  consider  subjects’  recommendations  regarding  whether  SnackTime 
should  switch  to  vitalia-formula  snack  foods  (see  Table  24).  An  analysis  indicates  a 
marginally  significant  two-way  interaction  on  switching  recommendations  (x  =2.96, 
p=085).  The  order  effect  is  significant  in  the  analogy-then-counter-reasons  cells 
(X2=9.93,  p=0016)  but  non-significant  in  the  reasons-then-counter-reasons  cells 
(X2=.73,  p=.39).  Moreover,  there  is  an  order  effect  in  favor  of  the  initial  analogy 
when  subjects  considered  multiple  analogies  (x2=7.41,  p=.0065).  An  analysis  of  the 
two-way  interaction  for  confidence  in  the  switching  recommendation  is  also 
significant  and  consistent  with  these  results  (Fi>n9=9.17,  p=.003).  Subjects  were 


more  confident  about  switching  when  considering  an  initial  positive  analogy  than  an 
initial  negative  analogy  followed  by  counter-reasons  (Fijh9=19.61,  p=. 0001);  the 
order  of  generating  pro  and  con  reasons  had  no  effect  on  confidence  (F<1).  In 
addition,  there  was  an  order  effect  when  subjects  considered  multiple  analogies 
(Fi, 119=12. 18,  p=. 0007).  Subjects  were  more  confident  when  they  considered  a 
success  analogy  followed  by  a failure  analogy,  compared  to  failure  then  success 
analogies. 

Second,  subjects  were  asked  to  provide  their  opinion  regarding  the  future  of 
vitalia-formula  products  at  SnackTime  (see  Table  25).  Subjects  rated  the  new 
product  on  four  scales:  “unlikely/likely  to  succeed,”  “will  sell  very  poorly  / very 
well,”  “not  a good  idea  / a very  good  idea,”  and  “do  not/definitely  recommend 
launch.”  A MANOVA  revealed  an  overall  two-way  interaction  (Fij  i7=6.27, 
p=.0136).  The  primacy  order  effect  was  stronger  in  the  analogy-then-counter-reasons 
cells  (Fi!h7=15.56,  p=. 0001)  than  the  reasons-then-counter-reasons  cells  (F<1).  In 
other  words,  subjects  were  more  optimistic  (pessimistic)  when  considering  an  initial 
positive  (negative)  analogy.  Counter-reasons  failed  to  eliminate  completely  the  bias 
induced  by  an  initial  analogy,  although  it  did  eliminate  the  effect  of  initial  reasons 
generated.  Moreover,  there  was  an  overall  primacy  effect  in  favor  of  the  initial 
analogy  when  subjects  considered  both  analogies  (F]jh7=4.48,  p=.0365).  In  other 
words,  counter-analogies  failed  to  eliminate  the  effect  of  initial  analogies.  This 
pattern  of  means  is  depicted  in  Figure  1 1 . 
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The  same  pattern  of  results  was  also  predicted  when  subjects  estimated  the 
percentage  of  Americans  regularly  consuming  vitalia-formula  snack  foods  over  time 
(see  Table  26).  Unexpectedly,  these  results  failed  to  reach  significance,  perhaps 
masked  by  noise  in  the  data.  There  was  a main  effect  of  order  (Fi  i i9=8.75,  p=.0037), 
but  it  was  not  qualified  by  an  interaction  with  the  consideration  of  analogies. 
However,  the  simple  main  effects  are  in  the  expected  direction:  the  order  effect  in  the 
analogy-then-counter-reasons  cells  is  significant  (Fi  n9=5.70,  p=.019)  but  only 
marginally  significant  in  the  reasons-then-counter-reasons  cells  (Fi,n9=2.63, 
p=  1 072).  However,  there  was  no  order  effect  when  subjects  considered  both 
analogies  (Fi,n9=T.03,  p=.313),  although  the  means  are  again  directional  (as  depicted 
in  Figure  12). 

An  analysis  was  also  conducted  on  the  reasons  generated  (see  Table  27). 
Subjects  were  asked  to  rate  the  reasons  that  they  generated  for  and  against  switching. 
Order  influenced  these  ratings  (FU]8=4.42,  p=.0377):  subjects’  ratings  were  higher 
for  reasons  in  the  first  set  (e.g.,  con  reasons  were  rated  as  more  persuasive  and  easier 
to  generate  when  they  were  considered  first  rather  than  second  ).  Subjects  were  also 
asked  to  count  the  reasons  that  they  generated  for  and  against  switching.  Order 
influenced  the  number  of  reasons  counted  (Fi  ng=12.16,  p=.007):  number  counts 
were  higher  for  reasons  in  the  second  set  (e.g.,  subjects  counted  more  pro  reasons 
when  they  were  considered  second).  This  pattern  is  interesting  because  the  greater 
number  of  reasons  in  the  second  set  was  seemingly  unable  to  compensate  for  the 
persuasiveness  of  the  initial  set  of  reasons  generated.  This  contrast  was  particularly 
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the  case  when  subjects  considered  an  initial  analogy  versus  reasons  (Fi  ng=2.59, 
p=.  1 10),  suggesting  that  an  initial  analogy  may  lend  persuasiveness  to  a set  of 
generated  reasons. 

Overall,  the  results  of  this  study  provide  considerable  support  for  the  notion 
that  there  are  order  effects  in  judgment  for  analogies  and  reasons.  First,  primacy 
order  effects  for  analogies  appear  to  be  stronger  than  for  reasons.  An  analogy  leads  to 
a ‘stickier  prior’;  that  is,  the  effect  of  an  initial  analogy  is  more  persistent  than  the 
effect  of  initial  reasons.  Second,  order  effects  endure  despite  counterfactual  thinking. 
In  contrast  to  prior  research  by  Hoch  (1984)  on  explanation  and  counterexplanation, 
counterfactual  reasoning  failed  to  eliminate  the  effect  of  an  initial  analogy.  In 
addition,  a counter-analogy  failed  to  eliminate  the  effect  of  an  initial  analogy.  This 
susceptibility  to  order  effects  raises  questions  about  the  efficacy  of  using  multiple 
analogies  to  improve  managerial  judgment.  Moreover,  the  consistent  results  in  this 
study  on  analogies  and  in  Study  1 on  scenarios  support  the  proposition  that 
nonanalytic  thinking  has  persistent  effects  on  judgment  that  are  difficult  to  eliminate 
with  subsequent  analytic  (or  nonanalytic)  counterfactual  thinking. 
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Table  24. 


Study  3:  recommendation  to  swtich  to  new  snack  food  product  and  confidence  in 
switching  as  a function  of  reasoning  order  and  analogy  cues  (New  Coke  and  Diet 
Coke). 


Code 

Type  of  Reasoning  and  Order 
1st  reasoning  task  2nd  reasoning  task 

Recommendations 
To  Switch 

Confidence 
in  Switching 

A1 

Positive  Analogy 
+ Pros 

Negative  Analogy 
+ Cons 

19/21 

72.14 

(13.03) 

A2 

Negative 
Analogy  + Cons 

Positive  Analogy 
+ Pros 

10/21 

50.00 

(25.10) 

AR1 

Positive  Analogy 
+ Pros 

Cons 

19/21 

68.81 

(18.23) 

AR2 

Negative 
Analogy  + Cons 

Pros 

8/21 

40.71 

(23.15) 

R1 

Pros 

Cons 

18/21 

64.76 

(18.54) 

R2 

Cons 

Pros 

15/20 

64.00 

(22.98) 

Table  25. 


Study  3 : attitude  ratings  as  a function  of  type  of  reasoning  and  order. 


Code 

N 

Unlikely  / 
Likely  to 
Succeed 

Will  Sell 
Very 
Poorly  / 
Very  Well 

Not  a Good 
Idea / A 
Very  Good 
Idea 

Do  Not  / 
Definitely 
Recommend 
Launch 

Average 

A1 

21 

4.71  (1.23) 

4.81  (1.21) 

4.90  (1.73) 

4.90(1.26) 

4.83  (1.26) 

A2 

21 

4.29(1.27) 

4.10(1.51) 

4.05  (1.63) 

3.95  (1.72) 

4.10(1.42) 

AR1 

20 

4.95  (0.94) 

4.85  (0.81) 

5.20(1.06) 

5.00  (1.30) 

5.00  (0.86) 

AR2 

21 

3.81  (1.33) 

3.76  (0.89) 

3.38(1.12) 

3.48  (1.33) 

3.61  (1.00) 

R1 

21 

4.71  (1.01) 

4.86  (1.06) 

5.00  (1.26) 

4.86(1.11) 

4.86  (0.97) 

R2 

19 

4.47(1.31) 

4.74  (1.33) 

5.00(1.15) 

4.68  (1.34) 

4.72(1.16) 

Note:  Code  definitions  (for  type  of  reasoning  and  order)  are  described  in  Table  24. 
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Order 
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Analogy 
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Counter 

Reasons 


Figure  11. 

Study  3:  attitude  toward  new  product  as  a function  of  type  of  reasoning  and  order. 


Table  26. 


Study  3:  predictions  of  regular  users  of  new  product  over  time  as  a function  of  type 
of  reasoning  and  order.  


Code 

N 

...  year 

Percentage  regular  users  in  next. . . 

...  2 years  ...  5 years  ...  1 0 years 

Average 

A1 

21 

25.00 

33.10 

38.67 

42.05 

34.70 

(21.67) 

(22.40) 

(24.17) 

(25.73) 

(18.83) 

A2 

21 

21.05 

28.90 

31.00 

34.14 

28.77 

(18.56) 

(21.13) 

(20.45) 

(25.81) 

(17.53) 

AR1 

21 

28.90 

38.52 

45.33 

56.24 

42.25 

(20.70) 

(17.95) 

(18.11) 

(25.49) 

(12.54) 

AR2 

21 

28.10 

27.10 

29.05 

33.88 

29.53 

(23.74) 

(14.10) 

(17.38) 

(25.24) 

(13.89) 

R1 

21 

25.62 

36.43 

49.19 

59.90 

42.79 

(20.54) 

(20.73) 

(23.36) 

(28.90) 

(20.40) 

R2 

20 

25.25 

31.13 

37.15 

42.60 

34.03 

(18.07) 

(17.26) 

(21.13) 

(29.13) 

(19.05) 

Note:  Code  definitions  (for  type  of  reasoning  and  order)  are  described  in  Table  24. 
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Figure  12. 

Study  3:  predictions  of  regular  consumers  of  snacktime’ s new  product  (averaged  over 
time)  as  a function  of  type  of  reasoning  and  order. 


Table  27. 


Study  3:  self-ratings  of  reasons  as  a function  of  type  of  reasoning  and  order. 


Code 

Net  reason  ratings  (ease  and 
persuasiveness  of  pros  minus  cons) 

Net  number  of  reasons 
(pros  minus  cons) 

A1 

0.24(1.19) 

-0.48  (1.50) 

A2 

-0.48  (2.09) 

0.00(1.87) 

AR1 

0.10(1.49) 

-0.85  (1.81) 

AR2 

-0.50(1.56) 

0.81  (1.33) 

R1 

0.00  1.35) 

-0.33  (1.39) 

R2 

-0.38  (1.01) 

0.45  (1.32) 

Note:  Code  definitions  (for  type  of  reasoning  and  order)  are  described  in  Table  24. 


CHAPTER  1 1 

STUDY  3A:  ANALOGIES  AND  COUNTER-ANALOGIES 
IN  A MARKETING  SIMULATION  ENVIRONMENT 


The  previous  study  demonstrated  that  counter-analogies  and 
counterexplanation  are  insufficient  to  neutralize  the  effect  of  an  initial  analogy.  This 
study  is  designed  to  address  the  use  of  multiple  analogies  in  a relatively  conservative 
test  environment.  First,  analogies  were  again  presented  to  subjects,  thereby 
eliminating  availability  biases  that  could  inhibit  the  generation  of  alternative 
analogies.  Second,  analogies  were  designed  to  be  irrelevant,  which  should  make  it 
easier  for  subjects  to  discount  or  argue  effectively  against  the  analogies.  Third, 
subjects  were  relatively  experienced  in  the  prediction  task  and  were  given  additional 
relevant  information  (market  research  data),  which  should  reduce  the  impact  of  less 
relevant  information  (such  as  analogies)  on  judgment.  And,  finally,  the  analogies  did 
not  include  any  explicit  numerical  information,  thereby  reducing  anchoring  as  an 
alternative  explanation  for  the  results.  Nonetheless,  my  hypotheses  were  consistent 
with  those  of  Study  3,  namely  that  the  schema  developed  under  an  initial  analogy  is 
difficult  to  undo.  That  is,  predictions  are  biased  in  the  direction  of  the  initial  analogy 
(primacy  order  effect),  and  alternative  analogies  do  not  debias  judgment. 
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Method 

Subjects  and  Design 

Subjects  were  undergraduate  and  a few  EMBA  students  who  participated  as 
part  of  a class  exercise.  Each  subject  was  randomly  assigned  to 
one  of  two  cells  in  a 2 (Analogy:  positive/negative)  x 2 (Repeat  predictions:  after 
initial  analogy  and  after  counter  analogy)  mixed  design.  A total  of  71  subjects 
participated,  with  the  number  in  each  cell  ranging  from  34  to  37. 

Materials  and  Procedure 

All  experimental  materials  took  the  form  of  surveys  distributed  to  each 
subject.  Subjects  worked  through  the  materials  individually  and  at  their  own  pace. 
The  survey  took  the  form  of  an  exercise  in  BrandMaps  (a  marketing  simulation 
game);  all  subjects  were  experienced  players/1  In  BrandMaps,  firms  compete 
against  each  other  in  a “vaporware  industry”  (a  fictitious  product  category  used  by  the 
simulation  game).  Firms  can  market  a varying  number  of  brands  in  each  region,  and 


Students  who  participated  in  this  study  were  experienced  in  the  BrandMaps  marketing  simulation 
game,  having  played  approximately  10  quarters.  Thus,  these  students  would  have  been  familiar  with 
the  fundamental  market  drivers  in  the  simulation  and  with  the  marketing  research  and  financial 
information  provided  to  them  in  BrandMaps.  In  this  regard,  use  of  the  BrandMaps  context  is  an 
attempt  to  examine  judgment  and  decision-making  with  so-called  ‘experts’  in  BrandMaps. 

The  majority  of  the  students  were  undergraduate  seniors  in  a marketing  management  class. 
Approximately  10%  were  Executive  MBAs  in  a marketing  management  class.  There  is  insufficient 
power  to  statistically  examine  differences  in  judgment  for  undergraduates  versus  executives  (as  a 
proxy  for  novices  versus  experts).  However,  there  are  no  directional  indications  that  executives  were 
any  less  biased  than  undergraduates. 

It  is  also  possible  to  examine  judgment  as  a function  of  subjective  (self-reported)  expertise.  Students 
were  asked  to  rate  themselves  on  several  aspects  of  expertise.  This  scale  was  factor-analyzed  and 
revealed  two  dimensions:  BrandMaps  expertise  (‘knowledge  of  decision  variables’,  ‘experience  using 
BM  research’,  contribution  to  my  team’s  performance’,  ‘attitude  toward  BM’)  and  Cola  market 
expertise  (‘knowledge  of  real-world  Pepsi-Cola’,  ‘knowledge  of...  New  Coke’,  ‘knowledge  of...  Diet 
Coke’).  Students  were  categorized  as  high  or  low  in  subjective  expertise  on  both  of  these  factor 
scores.  There  is  no  evidence  (directional  or  statistical)  that  subjective  experts  were  less  susceptible  to 
bias  than  novices. 
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students  are  responsible  for  the  launch,  development  and  marketing  of  their  firm’s 

brands.  Each  region,  firm  and  brand  has  a numerical  designation  (for  example,  firm 

2-1  refers  to  firm  #2  and  its  brand  #1).  Firms  and  regions  are  typically  named  in 

BrandMaps  (although  brands  are  not). 

In  the  BrandMaps  exercise,  subjects  were  given  background  information 

about  the  competitors  in  a target  region  and  their  own  firm’s  position  in  the  region. 

Imagine  that  you  are  part  of  the  management  team  for  “Coca-Cola 
Vaporware”  (Firm  1).  You  have  just  joined  the  firm  and  have  been  asked 
for  your  assessment  of  Coca-Cola  Vaporware’s  position  and  future  in  the 
U.S.  (Region  1).  Summary  statistics  for  Quarter  2 in  the  U.S.  region  are 
shown  below.  Briefly,  you  can  see  that  Coca-Cola  Vaporware  faces  2 
main  competitors  in  its  industry:  ‘Pepsi-Cola  Vaporware’  (Firm  2)  and 
‘No-Name  Vaporware’  (Firm  3).  Each  of  these  competitor  firms  has  one 
active  brand  in  the  U.S.  Coca-Cola  Vaporware  also  has  one  active  brand 
1-1  in  the  U.S.,  its  flagship  “Classic  Coke”  brand.  Classic  Coke  is  the 
leading  brand  in  terms  of  earnings  and  ROI  in  the  U.S. 

Subjects  were  told  that  Coca-Cola  Vaporware  is  considering  the  launch  of  a new 

brand  as  part  of  a project.  The  project  name  served  as  the  manipulation  of  the  initial 

analogy,  either  to  an  irrelevant  past  product  failure  or  success.  In  the  success  analogy 

condition,  students  were  told: 

Coca-Cola  Vaporware  has  a project  underway,  code-named  ‘Diet  Coke’, 
to  assess  the  feasibility  of  launching  a second  brand  1-2  in  the  U.S. 

Marketing  research  regarding  the  Diet  Coke  project— including  competitor 
and  customer  information  in  Region  1 , as  well  as  specific  research 
concerning  Diet  Coke-is  attached  (see  Table  of  Contents).  You  may  refer 
to  this  package  of  marketing  research  information  at  any  time  throughout 
this  exercise. 

In  the  failure  analogy  condition,  the  same  instructions  were  provided  with  the 


label  “New  Coke”  instead  of  “Diet  Coke”. 
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In  both  analogy  conditions,  students  were  given  detailed  information 
regarding  the  firm’s  marketing  plan  for  the  new  brand  (e.g.,  its  formulation, 
marketing  mix,  etc.)-  Students  also  received  the  same  firm  summary  statistics  and 
marketing  research  (e.g.  financial  reports,  concept  and  preference  tests  for  the 
new  brand,  test  market  data,  and  other  customer  and  competitor  information).  All 
students  experienced  in  the  simulation  would  be  familiar  with  this  information. 
Students  were  asked  for  a launch  recommendation  for  the  new  brand,  confidence 
in  this  recommendation,  and  predictions  of  market  shares  for  the  firms  in  the  next 
quarter  if  the  brand  were  launched. 

Students  were  then  asked  to  “briefly  describe  what  you  know  about  real- 

world  Coca-Cola’s  experience  with  Diet  Coke  and  New  Coke  in  the  soft  drink 

market.”  After  doing  so,  students  were  prompted  to  consider  both  the  analogy 

and  counter-analogy  in  an  evenhanded  manner  as  follows: 

You  may  or  may  not  find  real-world  Coca-Cola’s  experience  with  New 
Coke  and/or  Diet  Coke  in  the  soft  drink  market  useful  in  assessing  Coca- 
Cola  Vaporware’s  position  in  the  BrandMaps  vaporware  market.  Now 
that  you  have  had  a chance  to  give  the  matter  some  more  thought,  please 
answer  the  following  questions  related  to  the  project  launch  described 
earlier. 

Students  then  repeated  their  judgments  for  launch  recommendation,  confidence, 
and  market  share  predictions.  Students  also  answered  various  background 
questions  about  their  expertise  in  BrandMaps  and  knowledge  of  cola  markets. 

Results  and  Discussion 

Initial  predictions  were  biased  by  the  analogy  to  Diet  Coke  versus  New  Coke. 
Recommendations  for  launch  were  directionally  biased  in  the  direction  of  the  initial 
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analogy  (x2=2.39,  p=.  1 2).  Similarly,  confidence  in  launch  of  the  new  product  was 
marginally  higher  for  subjects  working  on  the  Diet  Coke  versus  New  Coke  project 
(Fi;69=3.18,  p=.079).  In  addition,  market  share  predictions  were  significantly 
different  for  the  existing  brand  Classic  Coke  in  the  market  (Fij66=5.38,  p=.023),  but 
not  for  the  new  brand  if  launched  (although  directionally  so).  The  differences  appear 
to  reflect  the  real-world  reasoning  that  New  Coke,  compared  to  Diet  Coke,  may 
cannibalize  sales  of  Classic  Coke.  This  is  evident  in  the  direction  of  the  means  in 
Table  28  and  was  also  mentioned  in  unanalyzed  cognitive  responses  of  subjects. 

After  students  were  prompted  to  think  about  both  New  Coke  and  Diet  Coke, 
their  repeated  predictions  continued  to  be  biased  by  the  initial  analogy.  The  effect  of 
the  analogy  on  recommendations  for  launch  and  confidence  in  this  decision  are  now 
both  significant  (respectively  x2=3.57,  p=.059;  Fi,63=9.03,  p=.004).  Market  share 
predictions  for  Classic  Coke  also  continued  to  marginally  differ  by  condition 
(Fij55=2.93,  p=.093);  the  difference  for  New  versus  Diet  Coke  remained  non- 
significant. Overall,  these  results  suggest  that  prompting  consideration  of  both 
analogies  did  not  eliminate  the  effects  of  the  initial  analogy. 

Another  view  of  the  data  considers  the  change  in  responses  with  repetition.  In 
the  intervening  period  between  first  and  repeated  judgments,  students  were  asked  to 
think  about  real-world  Coke’s  experience  with  both  Diet  Coke  and  New  Coke.  This 
intervention  serves  as  an  attempt  to  debias  judgment  by  prompting  evenhanded 
consideration  of  both  analogies.  As  we  have  already  seen,  the  analogy  does  bias  both 
initial  and  repeated  judgments,  so  this  attempt  at  debiasing  was  ineffective  overall. 
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However,  the  question  still  remains:  did  the  attempt  at  debiasing  have  at  least  some 
effect  on  improving  subsequent  judgment?  The  results  appear  mixed. 

1 . A MANOVA  of  both  measures  of  confidence  in  launch  of  the  new  product  as 
a function  of  condition  and  repetition  indicates  both  a main  effect  of  the  initial 
analogy  (Fij63=5.99,  p=.0 1 7)  and  an  interaction  with  repetition  (Fti63=6.33, 
p=. 014).  As  Figure  13  indicates,  instructions  to  think  about  both  analogies 
actually  exacerbates  the  pre-existing  bias,  presumably  by  prompting 
additional  analogy-driven  reasoning  beyond  a mere  anchoring  effect. 

2.  A MANOVA  of  both  Classic  Coke  market  share  predictions  as  a function  of 
condition  and  repetition  indicates  only  a main  effect  of  the  initial  analogy 
(Fi, 55-3. 52,  p=.066).  This  analysis  does  not  provide  any  indication  either  of 
exacerbation  or  amelioration  of  judgmental  bias.  The  biasing  effect  of  the 
initial  analogy  endures. 

3.  A MANOVA  of  both  New/Diet  Coke  market  share  predictions  as  a function 
of  condition  and  repetition  indicates  no  main  effect  of  the  initial  analogy 
(F<1)  but  a marginal  interaction  with  repetition  (F  155=2.  98,  p=.09).  This 
analysis  suggests  that  any  initial  difference  in  predictions  for  New  Coke 
versus  Diet  Coke  is  reduced  after  thinking  about  both  analogies. 

Overall,  the  data  seem  to  suggest  that  the  initial  analogy  continues  to  drive 
reasoning.  Confidence  in  the  launch  decision  drops  for  New  Coke,  but  does  not 
change  for  Diet  Coke.  Market  share  drops  for  New  Coke,  but  not  for  Diet  Coke; 
there  are  no  changes  in  predictions  for  Classic  Coke.  These  changes  seem  to  reflect 
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reasoning  by  analogy  to  the  real-world  experience  of  Coca-Cola,  which  suggests  that 
New  Coke  will  fail  but  Diet  Coke  will  succeed.  Interestingly,  the  major  impact  is 
via  the  analogy  to  New  Coke,  suggesting  that  spontaneous  consideration  of  failure  is 
otherwise  unlikely,  at  least  in  this  exercise.  It  must  be  said,  however,  that  this  is  also 
a calibration  issue  that  depends  upon  the  favorability  of  the  product  information 
provided  to  students.  If  this  information  is  sufficiently  positive,  there  may  be  a 
ceiling  effect  limiting  Diet  Coke’s  influence  on  judgment.  Because  the  design  lacks  a 
no-analogy  condition,  this  issue  cannot  be  addressed.  However,  the  notion  that 
prompting  people  to  consider  failure  particularly  improves  judgment  is  intriguing  and 
merits  further  investigation. 

The  findings  of  this  study  are  consistent  with  the  results  for  scenario  analysis 
in  Study  1.  Just  as  counter-scenarios  failed  to  eliminate  scenario-induced  bias,  so  did 
counter-analogies  fail  to  eliminate  bias  due  to  an  initial  analogy.  Test  conditions 
were  fairly  conservative  because  the  decision  environment  was  rich  in  analytic 
information.  In  conjunction  with  Pilot  Study  3,  these  results  provide  evidence  that 
counterfactual  reasoning  techniques  (generated  reasons,  presented  evidence,  and 
counter- analogies)  are  insufficient  to  debias  judgment.  In  the  next  study,  I continue 
to  investigate  the  effects  of  analogies  but  employ  decomposition  as  the  debiasing 


technique. 
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Table  28. 


Study  3A:  new  product  launch  decisions  and  forecasts  after  an  initial 
positive/negative  analogy  and  after  a counter-analogy. 


Condition 

Launch  - 

Confidence  in 

MS  #s  (%) 

MS#s  (%) 

(Initial  Analogy) 

Yes  / No 

Go  Decision 

New/Diet  Coke 

Classic  Coke 

Initial  Predictions  (given  an  analogy) 

Positive  Analogy 

35/2 

78.4 

19.7 

25.3 

(Diet  Coke) 

(95%) 

(16.8) 

(12.5) 

(9.1) 

Negative  Analogy 

28/6 

69.2 

23.2 

21.0 

(New  Coke) 

(82%) 

(26.0) 

(15.7) 

(5.4) 

Final  Predictions  (given  both  analogies) 

Positive  Analogy 

30/3 

76.7 

17.2 

25.4 

(Diet  Coke) 

(91%) 

(20.9) 

(11.7) 

(8.2) 

Negative  Analogy 

23/9 

58.0 

17.0 

22.3 

(New  Coke) 

(72%) 

(28.6) 

(12.2) 

(4.7) 

Change 

Change  in 

Change  in  MS  #s 

Change  in 

in  % 

Confidence  in 

(%) 

MS#s  (%) 

Launch 

Go  Decision 

New/Diet  Coke 

Classic  Coke 

Positive  Analogy 

-4% 

-0.76 

-1.0 

+0.7 

(Diet  Coke) 

(13.8) 

(5.5) 

(3.5) 

Negative  Analogy 

-10% 

-12.3 

-4.3 

+0.9 

(New  Coke) 

(22.5)* 

(8.6)  * 

(2.8) 

Note:  * indicates  change  is  significantly  different  from  zero.  However,  the  possibility 
of  drift  implies  that  contrasts  to  zero  are  an  inappropriate  analysis. 
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Figure  13. 

Study  3A:  confidence  in  new  product  decision  after  an  initial  positive/negative 
analogy  and  after  a counter-analogy. 


CHAPTER  12 

STUDY  4:  ANALOGIES  AND  DECOMPOSITION 


Research  on  decomposition  suggests  that  a ‘divide-and-conquer’  approach  to 
problem-solving  can  lead  to  improved  judgment.  Quantitative  estimation  tasks  have 
used  decomposition  to  break  down  problems  into  sub-components  that  can  be  more 
easily  and  accurately  estimated;  these  sub-components  are  then  recombined  into  an 
overall  estimation  of  the  target  quantity. 

In  a typical  multiplicative  decomposition,  an  algorithm  is  used  to  break  down 
a target  quantity  into  sub-components  that,  when  multiplied  together,  provide  an 
estimate  of  the  target  quantity.  For  example,  Armstrong,  Dennison,  and  Gordon 
(1975)  asked  subjects  to  predict  the  number  of  Polaroid  color  films  used  in  the  USA 
either  through  direct  estimation  or  through  decomposition  algorithms  provided  by  the 
experimenter  (e.g.,  number  of  families  in  USA,  percentage  of  families  owning 
cameras,  percentage  using  Polaroid  cameras,  average  number  of  films  packs  used, 
percentage  that  were  color).  The  authors  found  that  decomposed  estimates  were  more 
accurate  for  target  quantities  judged  highly  uncertain  or  difficult  to  estimate  directly. 
Subsequent  research  has  focused  on  two  aspects  of  multiplicative  decomposition:  the 
types  of  target  quantities  that  are  helped  by  decomposition  and  the  types  of 
algorithms  that  are  most  effective.  Research  suggests  that  decomposition  is 
especially  helpful  in  quantitative  estimation  of  extremely  large  uncertain  quantities. 
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For  example,  decomposition  improves  accuracy  for  target  quantities  with  extreme  and 
uncertain  values  (e.g.,  auto  accidents  per  year),  but  reduces  accuracy  for  smaller 
values  (e.g.,  US  presidents)  (MacGregor  & Armstrong  1994).  MacGregor, 
Lichtenstein,  and  Slovic  (1988)  also  found  that  accuracy  improved  most  in 
decomposition  for  problems  in  which  the  true  quantity  was  extremely  large  because 
direct  judgments  grossly  underestimate  these  quantities.  More  structured  algorithms 
(e.g.,  algorithms  including  instructions  on  how  to  re-combine  the  component 
quantities)  typically  led  to  greater  improvements  in  accuracy,  as  did  experimenter- 
provided  algorithms  compared  to  subject-generated  algorithms  (see  also  MacGregor 
& Lichtenstein  1991). 

Based  on  prior  research,  decompositional  algorithms  appear  to  be  effective 
analytic  tools  that  improve  judgment  compared  to  direct  estimation.  Looking  across 
these  studies,  the  main  benefit  of  decompositional  algorithms  may  arise  from 
encouraging  people  to  apply  analytical  reasoning  to  the  prediction  task. 
Experimenter-provided  algorithms  improve  accuracy  because  they  cue  relevant 
information  and  provide  a structure  for  analyzing  the  problem.  However,  prior 
research  has  not  investigated  whether  multiplicative  decomposition  is  equally 
effective  at  improving  judgment  that  is  already  biased  by  nonanalytic  thinking. 
Nonanalytic  thinking  may  create  prior  beliefs  (e.g.,  an  initial  ballpark  estimate)  that 
reduce  the  benefits  of  decomposition,  especially  if  bias  creeps  into  each  component  of 
the  algorithm  and  compounds  error. 


’2  Other  aids  to  managerial  judgment  may  share  this  approach  (e.g.,  decision  analysis). 
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I have  previously  argued  that  reasoning  by  analogy  may  drive  inference- 
making and  lead  to  one-sided  elaboration.  Biased  component  estimates  would 
provide  evidence  that  inference  making  has  been  biased  by  an  initial  analogy.  As  a 
result,  nonanalytic  thinking  that  creates  prior  beliefs  via  top-down  processing  (e.g., 
relying  on  an  analogy  to  provide  an  initial  ballpark  estimate)  may  reduce  the  benefits 
of  decomposition  by  creating  systematic  bias  in  the  decomposed  estimates,  which 
could  carry  over  to  the  recombined  estimates  as  well.  On  the  other  hand,  a 
decompositional  algorithm  may  prompt  bottom-up  processing  that  is  less  susceptible 
to  the  effects  of  an  initial  analogy. 

This  study  serves  as  a relatively  conservative  test  insofar  as  decompositional 
algorithms  are  provided  to  subjects  by  the  experimenter.  Debiasing  may  be 
particularly  ineffective  when  people  generate  their  own  algorithms  because  the  initial 
bias  can  influence  both  algorithm  generation  and  estimation.  For  this  reason,  the 
decompositional  algorithm  was  presented  to  subjects  in  an  attempt  to  prompt 
relatively  analytic  thinking.  The  study  is  also  designed  to  be  conservative  inasmuch 
as  the  product  analogy  is  meant  to  be  irrelevant.  The  analogous  product,  albeit  from 
the  same  company,  belongs  to  an  entirely  different  product  category  (radon  safety 
equipment  versus  child  safety  equipment).  Surface  similarity  was  manipulated 
through  the  name  of  the  product  in  an  attempt  to  alter  perceived  similarity.  If  this 
manipulation  succeeds,  then  the  initial  analogy  may  be  particularly  influential  when 
the  products  share  other  surface  similarities,  such  as  a similar  name. 
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Method 

Subjects  and  Design 

Subjects  were  undergraduate  students  who  participated  for  extra  credit  in  an 
introductory  marketing  class.  Each  subject  was  randomly  assigned  to  one  of  eight 
cells  in  a 2 (Analogy:  to  a product  with  high  versus  low  sales)  x 2 (Processing: 
bottom-up  versus  top-down)  x 2 (Product  name:  similar  versus  dissimilar)  between- 
subjects  design.  A total  of  92  subjects  participated,  with  the  number  in  each  cell 
ranging  from  10  to  12. 

Materials  and  Procedure 

All  experimental  materials  took  the  form  of  surveys  distributed  to  each 
subject.  Subjects  worked  through  the  materials  at  their  own  pace.  The  survey  asked 
subjects  to  imagine  that  they  were  the  new  marketing  manager  for  Home  Alert,  a 
manufacturer  of  safety  equipment  targeted  at  the  60  million  Americans  who  own  their 
own  homes.  The  survey  then  described  RadonAlert,  the  first  product  manufactured 
by  the  firm,  and  its  sales  and  market  share  (30  million  unit  sales  and  50%  market 
share  in  the  high  analogy  condition;  1.2  million  unit  sales  and  2%  market  share  in  the 
low  analogy  condition).  The  scenario  then  described  a new  product  idea  called 
ChildAlert  in  the  similar  name  condition  and  ChildMinder  in  the  dissimilar  name 
condition. 

In  the  top-down  processing  condition,  subjects  first  gave  an  initial  estimate  of 
market  share  and  unit  sales  of  this  product  in  the  first  3 years;  in  the  bottom-up 
condition,  subjects  proceeded  directly  to  the  decomposition  task.  The  decomposition 
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task  asked  subjects  to  estimate  intermediate  percentage  probability  estimates,  which 
form  the  basis  of  a decompositional  algorithm  for  estimating  probability  of  success. 
Subjects  then  provided  final  estimates  of  market  share  and  unit  sales  in  the  first  three 
years,  percentage  probability  that  the  product  will  succeed,  recommendation  to  launch 
product  or  not,  confidence  in  this  decision,  and  ratings  concerning  the  basis  for  the 
decision.  A final  open-ended  thought-listing  question  asked  subjects  why  this  product 
might  succeed  or  fail. 

Results  and  Discussion 

The  rationale  for  this  study  predicts  a three-way  interaction  between  name, 
analogy,  and  processing  on  predictions  for  final  unit  sales,  market  share,  probability 
of  success,  and  decision  to  launch  the  new  product,  as  well  as  on  the  intermediate 
probability  estimates.  An  ANOVA  indicated  that  this  interaction  was  not  significant 
for  any  of  the  dependent  measures,  with  the  exception  of  probability  of  success 
(Fi,83=4.75,  p=. 032)  where  the  results  do  not  appear  to  be  easily  interpretable  or 
theoretically  interesting  (see  Table  29  and  Figure  14).  Therefore,  the  surface 
similarity  manipulation  did  not  have  the  predicted  effects. 

The  analysis  now  turns  to  the  two-way  interaction  between  analogy  and 
processing  on  the  dependent  measures.  The  pattern  of  results  from  ANOVA  on  the 
continuous  dependent  measures  reveals  an  interaction,  although  CATMOD  and 
LOGIT  indicated  that  this  interaction  was  not  significant  for  the  binary  decision  to 
launch  variable.  The  two-way  interaction  between  analogy  and  processing  was 
significant  for  final  market  share  (Fi  79=2.99,  p=.088)  and  unit  sales  (F 1 71=5.82, 
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p=.01 8).  Because  of  the  significant  three-way  interaction  with  product  name, 
separate  analyses  were  conducted  for  probability  of  success  in  each  product  name 
condition.  The  simple  two-way  interaction  was  directional  for  probability  of  success 
in  both  conditions  (dissimilar:  Fi, 83=2.42,  p<.  13;  similar:  Fij83=2.33,  p<.  13).  As 
predicted,  the  analogy  had  no  effect  in  bottom-up  processing  (all  simple  effects  tests 
were  non-significant)  but  had  a significant  effect  in  top-down  processing  for  market 
share  (Fi,79=6.09,  p<.016),  unit  sales  (Fi>7i=14.37,  p<.001)  and  probability  of  success 
in  the  dissimilar-names  condition  (Fi, 83-10. 88,  p<.001)  and  similar-names  condition 
(Fi;83=8.92,  p<.004).  ’4  These  results  are  shown  in  Tables  29-30  and  Figures  14-15. 
Thus,  in  between-subjects  comparisons,  it  is  evident  that  subjects  who  provided  an 
initial  estimate  were  susceptible  to  analogy  effects,  and  their  judgment  was  not 
completely  debiased  by  task  decomposition.  In  contrast,  subjects  who  did  not  provide 
an  initial  estimate  but  instead  proceeded  directly  to  a decompositional  task  did  not 
appear  susceptible  to  analogous  product  information.  These  between-subjects  results 
are  consistent  with  a top-down  versus  bottom-up  processing  view. 

It  is  also  possible  to  examine  the  effects  of  the  decompositional  task  on  a 
within-subject  basis.  An  ANOVA  of  initial  and  final  market  share  and  unit  sales 
predictions  (measured  before  and  after  the  decomposition  task)  indicates  some  change 
in  subjects’  predictions.  For  market  share,  the  main  effect  of  analogy  (Fi,36=  8.47, 

^Analysis  of  unit  sales  must  be  interpreted  with  caution  due  to  variance  heterogeneity.  The  analysis  is 
consistent  with  ANOVAs  on  the  other  dependent  variables. 

34  There  were  also  effects  of  analogy  X name  (F  ! 79  =3 .97,  p<050)  and  analogy  (F 1 79  =3.3 1 , p=.073)  on 
market  share,  and  analogy  (F,  7!  =7.73,  p<.007)  and  process  (F)  7)  =3. 65, p=. 060)  on  unit  sales. 
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p=  006)  is  qualified  by  an  interaction  with  repeat  judgment  (Fi,36=4.45,  p=042).  For 
unit  sales,  there  is  only  a main  effect  of  the  analogy  (F  i ,37=  1 1.18,  p=.002).  Follow-up 
univariate  analyses  on  this  data  indicate  significant  effects  of  the  analogy  on  all 
estimates:  initial  market  share  (F 1 36=9.85,  p=.003),  initial  sales  (F  1,37=1 1.34, 
p=.002),  final  market  share  (F 1 36=6.78,  p=.013)  and  final  sales  (Fi,37=10.41,  p=.003). 
Thus,  although  repeat  predictions  are  affected  somewhat  by  the  decomposition  task,  it 
does  not  completely  eliminate  the  effects  of  the  initial  analogy. 

There  are  several  reasons  why  the  decomposition  task  may  be  ineffective  at 
debiasing  judgment.  First,  it  must  be  noted  that  the  decomposition  task  did  not 
provide  an  explicit  algorithm  instructing  subjects  how  to  combine  their  intermediate 
estimates  into  an  overall  judgment  (unlike  Pilot  Study  4).  If  subjects  were  not  forced 
to  reconcile  their  intermediate  estimates  with  their  final  judgments  and  were  free  to 
judge  and  interpret  the  meaning  of  these  intermediate  estimates  without  any  logical 
constraints,  then  some  of  the  benefits  of  the  decompositional  task  may  be  lost. 

Second,  it  is  possible  that  the  decompositional  task  itself  is  susceptible  to  analogy- 
driven  effects.  To  investigate  this  possibility,  a MANOVA  was  conducted  on  the 
intermediate  estimates  in  the  decomposition  task.  It  revealed  a marginal  main  effect 
of  the  analogy  (Fi, 74=3. 13,  p=.081),  indicating  that  intermediate  estimates  tended  to 
be  higher  in  the  high  versus  low  analogy  condition  (see  Table  31).  If  this  is  the  case, 
quantitative  estimates  calculated  from  decompositional  algorithms  could  also  be 
biased  and  perhaps  even  exacerbate  bias  if  systematic  error  is  inflated  due  to  the 
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nature  of  the  multiplicative  algorithm  (as  was  seen  in  the  calculated  estimates  in  Pilot 
Study  4). 

Consistent  with  the  notion  that  asking  for  an  initial  estimate  induces  strong 
anchoring  effects,  subjects  were  asked  to  rate  the  analogy’s  importance  in  their 
decision.  An  ANOVA  on  this  self-reported  measure  revealed  the  expected  main 
effect  of  process  (F i.8i=7.44,  p<.008).  Subjects  rated  the  analogy  as  more  important 
in  top-down  processing  than  in  bottom-up  processing  (see  Table  32).  There  was  also 
a significant  main  effect  of  product  name  (Fi,gi=5.56,  p<.021).  Subjects  rated  the 
analogy  as  more  important  when  the  product  names  were  similar  than  when  they  were 
dissimilar.  These  two  effects  suggest  that  perceived  need  or  usefulness  influences 
judgments  of  importance  of  information  (Greenberg,  Pyszczynski,  Warner,  & Bralow 
1994)  but  that  subjects  may  also  be  susceptible  to  irrelevant  similarities  such  as 
product  name.  However,  further  research  is  needed  to  follow  up  on  this  possibility. 

Overall,  the  most  important  finding  in  this  experiment  is  the  two-way 
interaction  between  the  analogy  and  type  of  processing.  As  expected,  asking  subjects 
to  provide  initial  market  share  and  sales  estimates  led  them  to  rely  on  analogous 
product  information.  The  effect  of  the  analogy  endured  despite  a subsequent  task  that 
asked  subjects  to  decompose  their  judgment  into  a sequence  of  conditional 
probability  estimates.  While  task  decomposition  did  have  some  corrective  effect  on 
judgment,  it  did  not  completely  eliminate  the  effects  of  the  initial  analogy,  as  evident 
in  comparisons  to  a bottom-up  processing  group  that  did  not  provide  initial  estimates. 
Two  conclusions  can  be  drawn  from  these  results.  First,  immediate  use  of  a 
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decompositional  task  in  quantitative  estimation  appears  to  induce  bottom-up 
processing,  thereby  limiting  the  effects  of  analogies  on  judgment.  Second,  making  an 
initial  estimate  appears  to  induce  top-down  processing  with  the  analogy-driven 
estimate  serving  as  a focal  hypothesis.  Once  people  adopt  the  analogy,  the 
effectiveness  of  debiasing  via  decomposition  is  reduced.  The  first  conclusion 
suggests  a boundary  condition  on  the  influence  of  analogies;  the  second  conclusion 
suggests  a boundary  condition  on  the  effectiveness  of  task  decomposition  in 
improving  judgment. 

As  I indicated  previously,  it  is  difficult  to  distinguish  analogy-induced  biases 
from  anchoring  effects  when  a numerical  anchor  is  included  with  the  analogy. 
Previous  studies  have  examined  analogy  effects  with  and  without  explicit  anchors. 
These  phenomena  also  may  be  naturally  confounded  much  of  the  time  in  the  real 
world.  If  so,  the  results  are  still  interesting  whether  seen  as  demonstrating  analogy  or 
anchoring  effects.  Prior  studies  also  demonstrated  that  several  analytic  tools  fail  to 
eliminate  nonanalytic  biases  in  judgment.  The  partial  effectiveness  of 
decompositional  techniques  merits  further  investigation.  On  the  one  hand, 
decompositional  techniques  are  influenced  by  reasoning  by  analogy  or  anchoring.  On 
the  other  hand,  decomposition  appears  to  limit  analogy  or  anchoring  effects  by 


inducing  bottom-up  processing. 
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Table  29. 


Study  4:  predictions  for  probability  of  success  as  a function  of  name,  analogy  and 
processing. 


Name 

Analogy 

Processing 

N 

Probability  of  Success 

Similar 

Low 

Top-down 

11 

43.64  (21.22) 

Similar 

Low 

Bottom-up 

11 

55.00  (27.02) 

Similar 

High 

Top-down 

10 

72.00  (26.69) 

Similar 

High 

Bottom-up 

12 

63.33  (18.38) 

Dissimilar 

Low 

Top-down 

11 

75.00(16.43) 

Dissimilar 

Low 

Bottom-up 

12 

57.25  (20.99) 

Dissimilar 

High 

Top-down 

12 

45.08  (23.46) 

Dissimilar 

High 

Bottom-up 

12 

47.08  (18.27) 

Table  30. 


Study  4:  predictions  of  new  product  market  share  and  unit  sales  as  a function  of 
analogy  and  processing. 


Ana- 

logy 

Process 

-ing 

N 

Final 

Market 

Share 

(%) 

N 

Final 

Unit 

Sales 

(millions) 

N 

Initial 

Market 

Share 

(%) 

N 

Initial 

Unit 

Sales 

(millions) 

High 

Top- 

21 

44.24 

22 

17.15 

20 

44.86 

21 

16.374 

Down 

(32.19) 

(23.70) 

(33.57) 

(22.76) 

High 

Bottom 

22 

37.86 

17 

4.94 

— 

— 

-up 

(32.43) 

(7.21) 

Low 

Top- 

21 

21.45 

20 

2.29 

22 

20.33 

22 

1.19 

Down 

(24.31) 

(3.05) 

(27.13) 

(1.376) 

Low 

Bottom 

23 

37.17 

20 

3.75 

— 

— 

iHE 

(33.65) 

(5.91) 
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Table  31. 


Study  4:  component  estimates  in  the  decomposition  task  as  a function  of  analogy. 


Component  Estimates  in  the  Decomposition  Task  (0-100% 
scale) 

High 

Analogy 

Low 

Analogy 

Percentage  of  homeowners  with  children 

60.0(17.6) 

63.3  (19.8) 

Percentage  of  homeowners  with  children  of  the  appropriate 
age  group 

36.6(15.5) 

38.0(18.5) 

Percentage  of  parents  concerned  about  their  child’s 
whereabouts 

77.2  (25.9) 

71.5  (27.4) 

Percentage  of  parents  who  would  consider  a monitoring 
system 

53.5  (25.1) 

45.6  (25.6) 

Percentage  of  parents  who  would  learn  of  the  existence  of 
ChildAlert/ChildMinder 

51.0(22.5) 

50.5  (24.8) 

Percentage  probability  that  competitors  will  offer  a similar 
product  in  the  next  3 years 

83.1  (18.7) 

70.0  (28.9) 

Percentage  of  consumers  who  would  prefer 
ChildAlert/ChildMinder  to  similar  competing  products 

60.7(17.5) 

53.0  (23.3) 

Percentage  of  stores  where  these  consumers  shop  who 
agree  to  carry  ChildAlert/ChildMinder) 

61.5  (25.5) 

53.3  (28.7) 

Average  of  probability  estimates 

60.5  (11.8) 

55.7(14.8) 

N 

37 

45 

Table  32. 

Study  4:  importance  rating  of  the  analogy  in  the  decision  to  launch  new  product. 


Processing 

Name 

N 

Rating 

Top-Down 

Similar 

20 

4.85  (1.82) 

Top-Down 

Dissimilar 

22 

3.82  (2.02) 

Bottom-Up 

Similar 

23 

3.67(1.61) 

Bottom-Up 

Dissimilar 

24 

2.79(1.93) 
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Study  4:  predictions  for  probability  of  success  as  a function  of  analogy  and 
processing  in  similar  (A)  and  dissimilar  (B)  name  conditions. 


Final  Market  Share  Estimates 
(after  decomposition  task) 


15 

10 

5 

o 

Low  Hgh 

Analogy 

— Bottom-Up  Top-Down 


Final  Sales  Estimates 
(after  decomposition  task) 


Analogy 


Bottom-Up  — ■—  Top-Dow  n 


A B 

Figure  15. 

Study  4:  market  share  (A)  and  unit  sales  (B)  as  a function  of  analogy  and  processing. 


CHAPTER  13 
GENERAL  DISCUSSION 

The  studies  reported  in  this  dissertation  provide  evidence  to  support  the  main 
propositions  of  this  dissertation. 

PI : Nonanalytic  thinking  about  scenarios  and  analogies  has  powerful  effects  on 

judgment. 

P2:  The  effects  of  nonanalytic  thinking  are  resistant  to  subsequent  analytic 

thought. 

The  results  of  a set  of  scenario  studies  support  these  propositions.  Pilot 
Studies  1 and  2 found  that  evenhanded  reasoning  does  not  eliminate  the  effects  of 
imagining  scenarios.  Thus,  the  effects  of  scenarios  merited  further  investigation. 
Study  1 demonstrated  that  a counter-scenario  does  not  eliminate  the  effect  of  an 
initial  scenario,  thereby  undermining  the  potential  effectiveness  of  scenario  analysis 
as  an  analytic  tool  for  improving  judgment.  Study  2 provided  evidence  that 
accountability  exacerbates,  rather  than  reduces,  the  effects  of  scenarios  on  judgment, 
suggesting  that  real-world  managers  will  not  be  immune  to  this  nonanalytic  bias. 
Turning  to  the  set  of  analogy  studies,  Pilot  Studies  3 and  4 demonstrated  that  deeper 
elaboration  via  scenarios  and  evenhanded  reasoning  does  not  eliminate  the  effects  of 
an  initial  analogy.  Thus,  the  effectiveness  of  other  analytic  tools  following  reasoning 
by  analogy  merited  further  investigation.  Studies  3 and  3A  demonstrated  that 
counter-analogies  fail  to  eliminate,  but  instead  may  exacerbate,  the  effect  of  an  initial 
irrelevant  analogy,  thereby  undermining  the  effectiveness  of  multiple  analogies 
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as  a judgment  tool.  Study  4 provided  evidence  that  decomposition  fails  to  debias 
judgment  that  is  already  influenced  by  an  initial  analogy.  However,  decomposition 
does  appear  to  reduce  reliance  on  reasoning  by  analogy,  although  the  decomposition 
task  itself  may  be  susceptible  to  the  analogy’s  effect.  Overall,  these  findings  attest  to 
robust  effects  of  nonanalytic  thinking  on  subsequent  analytic  judgment. 

Stickier  Priors 

The  most  important  finding  of  this  research  is  that  nonanalytic  thinking  leads 
to  stickier  priors.  In  Study  1,  judgments  were  biased  in  favor  of  an  initial  scenario, 
and  a counter-scenario  was  insufficient  to  eliminate  its  effect.  Importantly,  this 
primacy  order  effect  was  stronger  for  scenarios  than  reasons.  Similarly,  in  Study  3, 
judgments  were  biased  in  favor  of  an  initial  analogy  and  counterexplanation  was 
insufficient  to  eliminate  its  effect.  Again,  this  primacy  order  effect  was  stronger  for 
an  initial  analogy  than  for  initial  reasons  only. 

On  the  one  hand,  the  basic  finding  of  order  effects  in  judgment  is  nothing 
new.  Certainly,  there  is  a great  deal  of  evidence  to  suggest  that  people  are  non- 
Bayesian  and  experience  difficulty  in  generating  and  evaluating  multiple  hypotheses, 
as  I discussed  in  Chapters  2 and  5.  My  research  takes  this  finding  one  step  further, 
however,  and  provides  evidence  that  analytic  and  nonanalytic  thinking  are 
differentially  susceptible  to  order  effects  in  judgment.  People  favor  an  initial  scenario 
or  analogy  more  than  initial  reasons.  In  other  words,  the  initial  scenario  or  analogy 
leads  to  greater  fixation  or  a stickier  prior  than  more  analytic  reasoning.  But  why  is 


this  the  case? 
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As  a rationale  for  this  dissertation,  I proposed  that  imagination  of  scenarios 
and  reasoning  by  analogy  are  forms  of  elaboration  that  are  under  the  guidance  of  a 
focal  hypothesis.  Such  thought  is  unduly  influenced  by  associations  that  are  causally 
irrelevant  (e.g.,  mental  imagery  in  scenarios,  superficial  similarities  in  analogies).  In 
addition,  the  focal  hypothesis  inhibits  consideration  of  alternative  hypotheses.  As 
elaboration  continues,  a schema  develops  that  supports  the  focal  hypothesis  and 
drives  subsequent  thinking.  The  schema  drives  inference  making  and  leads  to  a well- 
integrated  set  of  beliefs  and  inferences,  in  contrast  to  discretely  encoded  information. 
A scenario-  or  analogy-driven  schema  that  is  relatively  coherent  and  consistent  tends 
to  be  unduly  convincing  and  influential;  that  is,  the  whole  is  greater  than  the  sum  of 
its  parts.  The  enriched  schema  increases  the  availability  and  perceived  plausibility  of 
the  focal  hypothesis,  thereby  influencing  subsequent  judgment.  This  one-sided 
elaboration  is  a source  of  mental  contamination  that  is  difficult  to  undo.  Although  it 
was  not  the  main  purpose  of  this  dissertation,  I can  nonetheless  review  evidence  in 
my  dissertation  concerning  this  rationale. 

Causally  irrelevant  information 

I have  previously  alluded  to  imagination  as  a unique  characteristic  of 
scenarios.  Pilot  Study  1 offered  evidence  that,  beyond  mere  explanation,  imagination 
(i.e.,  causally  irrelevant  details  and  mental  imagery)  has  powerful  effects  on 
judgment.  Similarly,  analogies  may  introduce  causally  irrelevant  information  (e.g., 
surface  similarities).  Although  irrelevance  is  in  the  eyes  of  the  beholder,  Study  3 A 
comes  closest  to  demonstrating  how  a relatively  subtle  and  irrelevant  analogy  cue  can 
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influence  judgment  despite  the  availability  of  considerable  analytic  information.  On 
the  other  hand,  my  (possibly  weak)  attempt  to  manipulate  the  surface  similarity  of  the 
analogy  in  study  4 failed  and  merits  further  investigation.  Interestingly,  the  danger  of 
causally  irrelevant  details  may  lie  in  their  seeming  irrelevance.  Such  details  serve  to 
embellish  and  fill  out  a schema  and  increase  its  perceived  plausibility,  yet  their  causal 
irrelevance  may  protect  these  same  details  from  critical  attention.  For  example, 
future  research  could  examine  whether  prompting  people  to  screen  or  assess 
analogies  reduces  the  influence  of  irrelevant  analogies.  In  addition,  a more  direct 
examination  of  the  role  of  causally  irrelevant  information  could  be  conducted  by 
manipulating  the  degree  to  which  scenarios  contain  mental  imagery. 

One-sided  elaboration  under  a focal  hypothesis 

Both  scenarios  and  analogies  appear  to  drive  one-sided  elaboration  under  a 
focal  hypothesis.  Indeed,  the  notion  of  elaboration  as  a fundamental  aspect  of 
scenarios  and  analogies  is  supported  by  my  research.  Pilot  Study  2 demonstrated  that 
elaboration  exacerbates  scenario  effects  on  judgment,  and  Pilot  Studies  3 and  4 
showed  that  analogies  endure  despite  further  elaboration.  Scenario  and  analogy 
effects  were  also  found  on  less  direct  judgment  measures,  which  can  be  interpreted  as 
evidence  of  inference  making  guided  by  the  initial  scenario  or  analogy.  For  example, 
scenarios  influenced  predictions  of  others’  attitudes  in  Pilot  Study  2 and  Study  1 and 
launch  recommendations  for  products  and  services  in  Study  1 . Similarly,  analogies 
influenced  component  estimates  in  the  decomposition  task  in  Pilot  Study  4 and  Study 
4.  There  was  also  some  (albeit  weak)  evidence  that  presentation  or  generation  of 
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evenhanded  evidence  may  lead  to  further  polarization  (Study  1,  Pilot  Study  3 and  4). 
In  other  words,  the  effects  of  analogies  and  scenarios  do  not  appear  to  arise  from 
relatively  superficial  reasoning  or  a lack  of  thought.  Overall,  these  studies  provide 
evidence  that  elaboration  contributes  to  scenario  and  analogy  effects  and  its  one- 
sidedness polarizes  judgments.  Future  research  about  how  people  generate  scenarios 
and  reason  by  analogy  is  needed  to  understand  what  constitutes  a good  or  compelling 
scenario  or  analogy  and  how  it  drives  subsequent  elaboration  and  judgment. 
Resistance  to  debiasing 

The  persistence  of  nonanalytic  thinking  may  lie  in  the  elaborate  and  integrated 
schema  that  develops  in  support  of  the  focal  hypothesis.  My  research  supports  the 
proposition  that  the  effects  of  scenarios  and  analogies  are  difficult  to  eliminate  with 
subsequent  analytic  thinking.  There  may  be  many  reasons  for  this.  First,  scenarios 
and  analogies  polarize  judgment  and  thereby  magnify  the  bias  that  must  be 
neutralized  with  subsequent  analytic  thinking.  For  example,  imagining  success  led  to 
more  optimistic  judgments  than  explaining  success  in  Pilot  Study  1 . Second,  once 
judgment  is  polarized,  it  may  bias  subsequent  analytic  thinking  and  reduce  its 
effectiveness.  For  example,  decomposition  was  ineffective  when  judgment  was 
already  affected  by  an  initial  analogy  in  Study  4.  Third,  tightly  integrated  and 
elaborate  schemas  may  be  more  difficult  to  undo  than  discretely  encoded  information 
(i.e.,  the  whole  is  greater  than  the  sum  of  its  parts).  My  present  work  can  offer  no 
direct  evidence  on  this  point,  although  it  is  consistent  with  prior  work  on  mental 
contamination  and  the  findings  of  this  dissertation.  Finally,  analytic  reasoning  may 
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simply  be  ineffective  at  countering  nonanalytic  thinking.  For  example,  Study  3 
suggests  that  analogies  remain  persuasive  despite  counterfactual  reasoning.  What 
would  be  an  effective  counterargument  for  a convincing  analogy  or  for  vivid  mental 
imagery  in  a scenario?  My  research  suggests  that  analytic  reasoning  is  not  effective 
nor  is  fighting  fire  with  fire  by  using  counter-scenarios  (Study  1)  or  counter-analogies 
(Study  3).  Further  research  is  necessary  to  fully  understand  the  processes  that 
contribute  to  the  power  of  scenarios  and  analogies  and  their  resistance  to  analytic 
thinking.  For  example,  how  does  initial  nonanalytic  thinking  affect  subsequent 
analytic  thinking,  and  how  do  people  combine  nonanalytic  and  analytic  information 
in  reaching  a final  judgment? 

Ineffectiveness  of  Analytic  Thinking 

While  this  research  has  demonstrated  powerful  effects  of  scenarios  and 
analogies  on  judgment,  it  has  also  shown  the  fallibility  of  various  analytic  tools 
heretofore  thought  to  improve  judgment.  Traditional  analytic  thinking  via 
evenhanded  reasoning  failed  to  neutralize  scenario  and  analogy  effects  on  judgment 
(in  all  pilot  studies).  Nor  did  counterfactual  reasoning,  counter-scenarios  (Study  1) 
and  counter-analogies  (Study  3).  The  failure  of  these  analytic  tools  does  not  appear 
to  be  entirely  memory-mediated  because  presented  evidence  also  failed  to  eliminate, 
and  may  even  have  exacerbated,  nonanalytic  effects  on  judgment.  Presented 
information  may  provide  grist  for  the  mill  and  become  integrated  into  the  scenario’s 
or  analogy’s  schema.  Similarly,  initial  nonanalytic  thinking  inhibited  the 
effectiveness  of  decomposition  (Pilot  Study  4 and  Study  4).  Top-down  processing 
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driven  by  the  initial  analogy  appears  to  dominate  subsequent  analytic  techniques  that 
strive  to  induce  bottom-up  processing.  Accountability  also  did  not  confer  immunity 
to  nonanalytic  thinking  and  seems  to  drive  people  to  think  harder,  not  smarter,  and 
further  elaborate  in  nonanalytic  ways  (Study  2).  Thus,  analytic  tools  are  susceptible 
to  nonanalytic  thinking  and  therefore  fallible,  a finding  that  deserves  emphasis  for  its 
contrast  to  prior  research. 

Construction  in  Judgment 

In  general,  predicting  the  future  reflects  a process  of  construction  rather  than 
retrieval.  One  constructs  the  future,  one  cannot  retrieve  it  from  memory.  It  is 
interesting  to  note  that  much  of  the  previous  work  on  generating  and  evaluating 
multiple  hypotheses  has  taken  place  in  fairly  analytic  contexts  that  emphasize 
retrieval  rather  than  construction  processes  (e.g.,  generating  pro  and  con  reasons; 
Hoch  1984).  If  differences  between  retrieval  and  construction  are  important,  it  may 
be  possible  to  reconcile  some  contradictory  findings  in  prior  research.  For  example, 
the  mere  salience  of  alternatives  or  the  breaking  of  a conditional  frame  of  reference 
(Koehler  1994;  Hirt  & Markman  1995)  may  be  sufficient  to  debias  judgment  when  it 
is  dominated  by  retrieval  processes.  One  need  only  bring  to  mind  other  alternatives 
to  obtain  improvements  in  judgment.  Flowever,  something  beyond  mere  salience 
may  be  required  to  debias  judgment  that  is  based  on  construction  processes.  The  act 
of  construction  may  itself  lead  to  a schema  that  is  difficult  to  de-construct  or  undo,  or 
that  makes  an  alternative  construction  difficult  to  undertake.  The  uncertainty  of  a 
prediction  environment  makes  it  difficult  for  people  to  be  deliberately  evenhanded 
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and  analytic,  thereby  increasing  reliance  on  nonanalytic  thinking.  Thus,  the  process 
of  constructing  the  future  seems  especially  susceptible  to  one-sided  elaboration 
induced  by  a scenario  or  analogy. 

In  essence,  a scenario  tells  a story  about  the  future.  In  doing  so,  it  helps 
people  to  organize  and  make  sense  of  information.  Similarly,  an  analogy  also  helps 
people  to  organize  and  make  sense  of  information  by  providing  a focal  hypothesis 
that  guides  subsequent  thinking.  Very  little  may  be  needed  to  induce  one-sided 
elaboration  during  the  construction  process  (e.g.,  Holyoak  & Simon  1999).  Indeed, 
an  initial  leaning  that  develops  in  one  direction  or  another  may  start  people  down  the 
path  of  scenario  generation  and  elaboration  that  ultimately  leads  to  polarized 
judgment.  Of  course,  other  factors  may  also  start  elaboration  in  a particular  direction. 
For  example,  an  irrelevant  analogy  embedded  in  new  product  information  may  drive 
subsequent  thinking  and  thereby  polarize  judgment.  Similarly,  situational  factors 
may  exacerbate  one-sidedness  in  thinking  (e.g.,  accountability  and  other  incentives). 
Although  there  may  be  some  motivation-based  preference  for  coherent  and  consistent 
schemas  and  judgments,  my  research  suggests  that  one-sidedness  arises 
spontaneously  as  a result  of  the  cognitive  processes  of  scenario  generation  and 
reasoning  by  analogy. 

Other  Issues 

Expertise 

The  studies  described  in  this  dissertation  were  conducted  in  a controlled 
laboratory  environment  using  undergraduate  students  as  subjects.  Some  readers  may 
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wonder  whether,  unlike  students,  managers  would  be  immune  to  the  potentially 
harmful  effects  of  scenarios  and  analogies.  Managers  may  differ  from  students  on 
important  characteristics.  For  example,  managers  have  expertise  and  are  held 
accountable  for  their  judgments.  These  differences  are  presumed  by  some  to  confer 
immunity  to  error.  My  scenario  studies  suggest  that  accountability  does  not  reduce 
susceptibility  to  error,  so  I will  confine  my  remarks  in  this  section  to  expertise. 

First,  new  product  innovation  is  an  area  in  which  expertise  is  difficult  to 
develop.  The  failure  rates  of  new  product  and  businesses  attest  to  this  lack  of 
expertise.  By  their  very  nature,  new  products  are  new  and  relatively  unique  and 
learning  through  experience  is  made  difficult  by  the  complexity  and  timeline  of  new 
product  decisions.  Amidst  this  difficulty  and  uncertainty,  scenarios  and  analogies 
may  be  particularly  seductive  as  a guide  to  judgment.  Second,  the  effects  of  scenarios 
and  analogies  are  attributable  to  psychological  processes  that  expertise  does  not 
fundamentally  alter.  Prior  research  has  demonstrated  that  managers  are  not  immune 
to  other  judgment  errors  (e.g.,  the  sunk  cost  effect;  Boulding,  Morgan,  & Staelin 
1997).  It  is  difficult,  then,  to  argue  for  immunity  in  the  case  of  analogies  and 
scenarios. 

Two  studies  were  designed  to  demonstrate  the  effects  of  scenarios  and 
analogies  with  experts.  In  the  first  study  (reported  in  Study  3A),  both  executive  and 
undergraduate  students  gained  experience  in  a marketing  simulation  game. 
Nonetheless,  subjects’  forecasts  for  the  success  of  a new  product  were  influenced  by 
an  initial  irrelevant  analogy.  Neither  their  previous  experience  nor  extensive  analytic 
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evidence  provided  in  marketing  research  data  nor  a counter-analogy  presented  to 
subjects  was  sufficient  to  eliminate  the  effects  of  the  initial  analogy  on  their 
judgment.  In  a second  study,  MBA  students  who  have  completed  an  e-commerce 
course  will  be  asked  to  use  scenario  analysis  to  forecast  the  future  of  a new  Internet 
business  venture.  The  purpose  of  this  study  is  to  demonstrate  that  initial  scenarios 
dominate  judgment  even  for  these  trained  experts  and  even  when  counter-scenarios 
are  generated.  At  the  time  of  writing,  this  study  was  under  preparation.  Of  course, 
experts  may  generate  only  a single  success  scenario,  which  may  account  for 
entrepreneurial  optimism  in  the  face  of  low  base  rates  for  success. 

Groups 

This  dissertation  focused  on  individual  judgment.  Some  readers  may  wonder 
whether,  unlike  individuals,  groups  would  be  immune  to  the  potentially  harmful 
effects  of  scenarios  and  analogies.  Interestingly,  group  research  offers  evidence 
consistent  with  the  notion  of  one-sided  elaboration  and  also  suggests  that  groups  will 
not  be  immune  to  these  judgmental  biases. 

Group  research  points  to  the  importance  of  construction  processes  in 
judgment  and  the  danger  of  one-sided  thinking.  For  example,  both  interpersonal 
influence  and  repeated  expression  of  one’s  own  attitude  contribute  to  group 
polarization,  which  suggests  that  group  polarization  is  basically  a group  explanation 
bias  (Brauer,  Judd,  & Gliner  1995).  Repeated  expression  was  particularly  powerful 
when  expressed  arguments  were  picked  up  and  repeated  by  other  group  members, 
which  is  similar  to  exposure  to  one’s  own  arguments  again  in  presented  evidence.  In 
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a similar  vein.  Heath  and  Gonzales  (1995)  propose  that  the  key  to  understanding 
group-induced  biases  is  whether  the  group  interaction  promotes  or  hinders  “rationale 
construction”  (whereby  people  organize  and  articulate  their  thoughts  to  support  their 
opinion,  a form  of  explanation  bias).  Predictions  made  by  individuals  after 
interactions  with  group  members  were  no  more  accurate  yet  were  more  confident; 
similarly,  a “directed  writing”  task  also  increased  confidence  whereas  “pro/con 
writing”  did  not.  These  results  are  consistent  with  the  notion  that  people  prepare  to 
support  their  opinion  to  the  group  and  this  explanation  task  biases  subsequent 
judgment.  Presented  opposing  arguments  (i.e.,  counterarguing  by  others  in  the  group) 
had  little  impact  on  individual  judgment;  however,  actively  generated  pro  and  con 
reasons  in  a writing  task  did.  Presumably,  subjects  who  were  instructed  to  engage  in 
pro/con  writing  were  encouraged  to  pay  attention  to  and  generate  both  supporting  and 
opposing  arguments,  thereby  preventing  rationale  construction  and  polarization  of 
judgment. 35 

Rationale  construction  is  reminiscent  of  story  construction,  whereby  people 
construct  stories  to  explain  outcomes.  Rationale  construction  may  be  a process 
similar  to  scenario  generation  and  reasoning  by  analogy,  inasmuch  as  it  induces  one- 


° Research  investigating  dyadic  interaction  is  also  consistent  with  this  notion.  Sniezek  and  Henry 
(1989)  found  that  groups  were  more  confident  and  more  accurate  in  estimating  event  frequencies  (e.g., 
for  causes  of  death),  which  the  authors  attribute  to  increased  cognitive  effort.  Allwood  and  Granhag 
(1996)  found  that  dyad  members  who  answered  almanac  questions  separately  and  then  together 
exhibited  lower  overall  overconfidence  in  their  group  versus  individual  answers.  Confidence  rose 
when  pair  members  agreed  and  fell  when  pair  members  disagreed,  suggesting  that  counter-explanation 
was  at  least  somewhat  effective  (Allwood  & Granhag  1996).  The  authors  speculate  on  the  importance 
of  forcing  subjects  to  utilize  their  knowledge  and  engage  in  discussion,  thereby  confronting 
counterarguments  (in  contrast  to  Allwood  & Bjoerhag,  1990). 
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sided  elaboration  under  a focal  hypothesis,  thereby  increasing  its  plausibility  and 
biasing  judgment. 

Managerial  Implications 

As  new  product  innovation  becomes  increasingly  important  as  a driver  of 
sales  and  growth,  it  is  critical  for  firms  to  identify  successful  new  product  ideas  and 
forecast  their  future.  The  findings  from  this  dissertation  have  a number  of 
implications  for  managers  who  work  on  the  development  and  introduction  of  new 
products.  In  particular,  the  early  stages  of  thinking  about  new  products  are  critical 
because,  once  judgments  are  formed,  they  are  difficult  to  change.  Judgments  are 
particularly  sticky  when  based  on  scenarios  and  analogies  (compared  to  analytic 
reasoning).  Thus,  scenarios  and  analogies  should  be  used  with  caution  and  subjected 
to  greater  scrutiny  in  new  product  forecasting,  especially  in  the  early  stages  of  new 
product  development.  The  use  of  analytic  tools  early  in  the  forecasting  process  may 
reduce  the  inhibiting  effects  of  nonanalytic  thinking.  At  the  very  least,  managers 
should  reduce  one-sidedness  in  judgment  by  using  multiple  counter-analogies  and 
counter-scenarios,  rather  than  allowing  one  specific  analogy  or  scenario  to  dominate 
thinking.  Future  research  could  also  investigate  screening  processes  for  the  adoption 
of  analogies  and  scenarios. 

The  findings  from  this  dissertation  also  suggest  that  analytic  tools  may  be  less 
effective  when  they  follow  nonanalytic  thinking.  Evenhanded  reasoning,  presented 
evidence,  counter-analogies,  counter-scenarios,  accountability  and  decomposition  all 
failed  to  neutralize  the  effects  of  initial  nonanalytic  thinking.  Once  scenarios  and 


150 


analogies  creep  into  the  judgment  process,  they  are  difficult  to  eliminate  with  analytic 
thinking.  Managers  should  beware  of  overconfidence  in  these  (fallible)  analytic 
tools.  Moreover,  managers  can  also  assess  organizational  structure  and  processes  for 
their  susceptibility  to  nonanalytic  thinking  and  modify  the  use  of  analytic  tools  to 
improve  their  effectiveness.  For  example,  my  dissertation  suggests  that  managerial 
involvement  in  the  generation  of  scenarios  biases  their  subsequent  evaluation. 
Disaggregation  of  these  stages  across  managers  may  lead  to  improved  forecasting. 
Similarly,  changes  in  the  product  development  process  to  separate  nonanalytic 
thinking  from  subsequent  analytic  assessment  should  be  examined.  Although  early 
design  and  development  work  may  rely  heavily  on  scenarios  and  analogies  as  a spur 
to  creativity,  their  influence  may  be  less  beneficial  in  forecasting  and  launch 
decisions  that  require  more  analytic  thinking. 

In  addition,  these  findings  on  scenarios  and  analogies  may  be  applied  to  other 
areas  of  business  and  marketing,  including  planning  and  strategic  decision-making. 
Plans  share  characteristics  with  scenarios:  they  describe  in  detail  how  the  future  will 
unfold  and  provide  a blueprint  for  what  the  firm  will  do.  As  such,  plans  are 
susceptible  to  one-sidedness  and  will  lead  to  sticky  judgments  that  resist  negative 
feedback  and  efforts  to  change  them.  This  stickiness  could  account  for  the  planning 
fallacy,  a tendency  toward  optimism  with  plans  (e.g.,  Buehler  et  al.  1994;  Connolly  & 
Dean  1997).  For  example,  plans  that  fail  to  fully  take  into  account  all  the  things  that 
can  go  wrong  (i.e.,  worst-case  scenarios)  will  underestimate  new  product 
development  and  launch  times  as  well  as  needed  resources.  Strategic  decisions,  like 


new  product  decisions,  are  novel  and  complex  and  may  rely  on  soft  judgment  guided 
by  scenarios  and  analogies.  For  example,  the  merger  of  two  firms  is  difficult  to 
assess  analytically,  and  analogies  and  scenarios  for  the  success  of  a merger  may  be 
tantalizingly  easy  to  generate.  Witness  recent  upside  scenarios  and  analogies  to 
General  Motors  Corporation  regarding  the  Time  Warner-AOL  merger  (Business 
Week,  24  Jan  2000).  The  potential  downside  of  nonanalytic  thinking  in  strategic 
decision  making  and  planning  merits  attention. 

Consumer  use  of  scenarios  and  analogies  in  responding  to  new  product 
concepts,  predicting  their  own  behavioral  intentions,  and  planning  for  their  own 
futures,  also  merits  further  investigation.  Scenarios  and  analogies  may  be  powerful 
persuasive  tools  with  which  to  sell  new  products  to  customers  (or  other  constituents 
within  or  outside  the  firm).  Indeed,  Gregory  and  Duran  (2000)  discuss  how 
consultants  can  use  scenarios  as  a persuasive  tool  to  increase  client  acceptance  of 
forecasts  and  client  commitment  to  strategic  plans.  Rather  than  telling  customers 
why  they  should  buy  a new  product,  firms  can  tell  them  a story  about  how  they  will 
use  a new  product  or  provide  them  with  a favorable  analogy.  For  example,  Hewlett- 
Packard  has  created  a videotape  scenario  describing  the  wireless  future  of  health  care 
in  hospitals,  and  new  toys  are  often  touted  as  the  next  Tickle-me-Elmo  to  persuade 
stores  to  stock  them  and  customers  to  buy  them.  Once  customers  develop  an  initial 
judgment  based  on  a scenario  or  analogy,  this  judgment  may  be  fairly  resistant  to 
subsequent  analytic  thinking.  Focus  groups  and  concept  testing  can  be  designed  to 
elicit  the  scenarios  and  analogies  favored  by  customers  for  use  in  subsequent 
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marketing  efforts.  The  use  of  favorable  analogies  and  scenarios  may  boost  customer 
adoption  rates  for  new  products  and  encourage  loyalty  to,  and  reduce  defection  from, 
existing  brands.  In  this  way,  firms  may  be  able  to  take  advantage  of  nonanalytic 
thinking. 
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GENERAL  AUDIENCE  ABSTRACT 


Judgmental  forecasting  is  an  inherent  component  of  new  product  development 
and  strategic  decision-making.  Managers  and  firms  often  struggle  with  these 
decisions,  as  evidenced  by  failure  rates  of  new  products  and  businesses.  This 
dissertation  investigates  sources  of  bias  in  new  product  forecasting. 

I identify  two  potential  sources  of  bias:  imagining  scenarios  and  reasoning  by 
analogy.  In  a series  of  laboratory  studies,  I demonstrate  that  an  initial  scenario  or 
analogy  has  a persistent  effect  on  judgment.  Subsequent  efforts  to  be  more  analytic 
(by  generating  reasons,  or  considering  counter-scenarios  or  counter-analogies,  or 
using  other  analytic  tools)  fail  to  neutralize  the  effect  of  a scenario  or  analogy. 

In  other  words,  a scenario  or  analogy  creates  a ‘stickier  prior’  than  a judgment 
based  on  traditional  reasoning.  Misleading  scenarios  or  analogies  may  lead  managers 
to  make  inaccurate  forecasts.  The  persistence  of  these  forecasts  despite  subsequent 
analytic  thinking  may  account  for  some  of  the  errors  in  new  product  forecasting. 


